llama.cpp
The C++ inference engine powering most local LLMs.
Open SourceCPU-capable
Actually FreeNo SignupOpen SourceWatermark-Free
Transformers
The library every LLM ships against first.
Open Source 2–24 GB VRAMRuns locally
Actually FreeNo SignupOpen SourceWatermark-FreeAPI
Visit TransformersUpdated 2026-05-21 · Direct link
Hardware requirements
Runs locally · Entry GPU (6–8 GB)
Min VRAM
2 GB
Rec. VRAM
24 GB
Min RAM
8 GB
Rec. RAM
32 GB
Disk
30 GB
GPU class
Entry GPU
11.8+Apple Silicon ✓CPU-CapableQuant: FP16, BF16, INT8 +2
Scales with the model you load. Tiny LMs run CPU; 70B-class needs 24 GB+ quantized.
Screenshot placeholder · Transformers
What is Transformers?
Hugging Face Transformers is the universal Python library for loading and running LLMs (and vision, audio, multimodal models). Slower than vLLM / llama.cpp for serving, but it's where every new model lands first, and the API is the closest thing the field has to a standard.
Pros & cons
Pros
- Universal loader — every new model lands here first
- Same API across architectures (LLaMA, Mistral, Qwen, Gemma, etc.)
- First-class quantization (bitsandbytes, GPTQ, AWQ, FP8)
- Trainer + PEFT for fine-tuning included
Cons
- Slower inference than vLLM / llama.cpp at production scale
- Library, not a server — you write the orchestration
What's actually free?
Apache 2.0 from Hugging Face.
✓ Actually FreeNo SignupOpen SourceWatermark-Free
