Best local models for llama.cpp

34B class • 15.5 GB minimum

Community GGUF packaging gives llama.cpp a direct path.

34B class • 19.5 GB minimum

Community GGUF import path for llama.cpp.

3B class • 2.5 GB minimum

IBM's smallest Granite 4.0 instruct release is a pragmatic US-origin starter for local chat, extraction, and agent scaffolding.

7B class • 5.0 GB minimum

Ai2's 7B instruct release is the clearest Apache-licensed American alternative to Llama when you want a smaller fully open local model.

7B class • 6.5 GB minimum

Meta's 8B instruct release remains the safest broad-compatibility US local model when you want maximum runtime coverage.

13B class • 8.5 GB minimum

Phi-4-reasoning is the clearest text-first American recommendation around the 13B class when you care about reasoning quality more than multimodal extras.

13B class • 11.0 GB minimum

Gemma 3 12B stays interesting when you want a smaller multimodal American model, but it is less turnkey than Phi-4-reasoning for plain text work.

34B class • 19.5 GB minimum

Granite 4.0 H-Small is a credible American midrange choice for RAG-heavy work, but it is more specialized than the general-purpose winners above it.