llama.cpp
by ggml · Runtime · Text & code
promising
Widely used local runtime reaching 1.0 with stable APIs; maintainer load is a concern.
v1.0.0 declares API stability, removes 35 deprecated public-header functions, and adds a hot-expert cache for MoE models larger than GPU memory
Why we say so
ggml-org says 1.0 declares API stability, removes 35 deprecated functions and adds a hot-expert cache for large MoE models. Fahd Mirza used it to serve a 35B model locally without trouble. r/LocalLLaMA commenters worry about the pile of open issues and PRs and want more maintainers.
Evidence
- Nov 18Commenters worry about many open issues and PRs and ask for more maintainers; others ask for upstream MoE streaming techniques
- Nov 13v1.0.0 released; libllama and libggml adopt semantic versioning, with major bumps for API/ABI breaks
- Sep 8Running GLM 5.3 Flash needs a special llama.cpp branch with GLM 5.3 support; a stock existing install will not work
- Sep 1Praised for one-binary deployment, heterogeneous hardware and wide quantization support, but called slow for concurrent use
- Aug 15Reviewer prefers plain llama.cpp over the Ollama and LM Studio wrappers; lower memory use once context is reduced
Show all 7
- Aug 9Kimi K3 is not merged into stock llama.cpp; runs from a pull-request branch and CPU-only loading crashes, so a GPU is needed
- Jul 18Needs the --jinja flag for Gemma 4 tool calls; without it tool definitions are ignored and calls return as plain text
Runs on llama.cpp
18 models · newest firstAlternatives
Made for llama.cpp