llama.cpp
The C++ inference engine powering most local LLMs.
Open SourceCPU-capable
Actually FreeNo SignupOpen SourceWatermark-Free
Ollama
One-command local LLM runtime.
Open SourceCPU-capableRuns locally
Actually FreeNo SignupOpen SourceWatermark-FreeHobbyist-FriendlyAPI
Visit OllamaUpdated 2026-05-15 · Direct link
Hardware requirements
Runs locally · Mid GPU (12 GB)
Min VRAM
None
Rec. VRAM
16 GB
Min RAM
8 GB
Rec. RAM
32 GB
Disk
50 GB
GPU class
Mid GPU
CUDA optional (NVIDIA)Apple Silicon ✓CPU-CapableQuant: Q4_K_M, Q5_K_M, Q6_K +2
7B Q4 runs on 8 GB. 13B Q4 on 12 GB. 70B Q4 needs 48 GB unified or split GPU.
Screenshot placeholder · Ollama
What is Ollama?
Ollama wraps llama.cpp behind a clean CLI and HTTP API. Pull a model (`ollama run llama3.1`), get a chat or an OpenAI-compatible endpoint. Excellent default for hobbyists running quantized models.
Pros & cons
Pros
- Easiest local LLM onboarding
- OpenAI-compatible API
- Great Apple Silicon performance
Cons
- Opinionated model registry
- Less control than raw llama.cpp
- Single-user by default
What's actually free?
Fully free / OSS. You provide the hardware.
✓ Actually FreeNo SignupOpen SourceWatermark-Free
Alternatives
LM Studio
Desktop GUI for running local LLMs.
FreemiumCPU-capable
Actually FreeNo SignupWatermark-FreeHobbyist-Friendly
Text Generation WebUI
The "A1111 for LLMs" — multi-loader local chat UI.
Open Source 6–24 GB VRAM
- Min VRAM
- 6 GB
- GPU class
- High-end GPU
- Quant
- GGUF
Actually FreeNo SignupOpen SourceWatermark-Free
vLLM
High-throughput LLM serving for GPUs.
Open Source 24–80 GB VRAM
- Min VRAM
- 24 GB
- GPU class
- Datacenter GPU
- Quant
- FP16
Actually FreeNo SignupOpen SourceWatermark-Free