Ollama
One-command local LLM runtime.
Open SourceCPU-capable
Actually FreeNo SignupOpen SourceWatermark-Free
llama.cpp
The C++ inference engine powering most local LLMs.
Open SourceCPU-capableRuns locally
Actually FreeNo SignupOpen SourceWatermark-FreeHobbyist-FriendlyAPI
Visit llama.cppUpdated 2026-05-17 · Direct link
Hardware requirements
Runs locally · High-end GPU (16–24 GB)
Min VRAM
None
Rec. VRAM
24 GB
Min RAM
8 GB
Rec. RAM
64 GB
Disk
100 GB
GPU class
High-end GPU
CUDAApple Silicon ✓CPU-CapableQuant: Q2_K, Q3_K, Q4_K_M +6
Mixed CPU+GPU offload is its superpower for 70B+ on a single card.
Screenshot placeholder · llama.cpp
What is llama.cpp?
The reference CPU/GPU inference engine for GGUF-quantized LLMs. Ollama, LM Studio, Jan, and KoboldCpp all sit on top of it. Use it directly when you need raw control, multi-GPU split, or exotic quants.
Pros & cons
Pros
- Best quantization support anywhere
- Runs on basically anything
- Multi-GPU tensor split
Cons
- CLI-first
- You manage models yourself
What's actually free?
Free / OSS.
✓ Actually FreeNo SignupOpen SourceWatermark-Free
Alternatives
vLLM
High-throughput LLM serving for GPUs.
Open Source 24–80 GB VRAM
- Min VRAM
- 24 GB
- GPU class
- Datacenter GPU
- Quant
- FP16
Actually FreeNo SignupOpen SourceWatermark-Free