Ollama
Runtime · Text & code
promising
A usable local backend, but small prompt changes can wipe its prefix cache.
Found that one system-prompt timestamp or a change in reasoning effort can wipe the prefix cache
AccessOpen codePriceFree to run locally; cloud Pro $20/month, Max $100/monthRuns onWindows, macOS, LinuxLicenseMIT
Why we say so
Simon Willison names Ollama as a local OpenAI-compatible backend that works with datasette-llm. Ollama's own testing found that a single system-prompt timestamp, or a change in reasoning effort, can wipe the prefix cache and slow repeated requests. There are no broader reviews here.
Evidence
- Nov 14Named as a local OpenAI-compatible backend usable with datasette-llm
- Sep 4Used via its API with Qwen 3.6 27B (needs tool use) on an RTX A6000 48 GB to summarize Obscura output
- Aug 26Works as a custom provider in DSH via Ollama's default endpoint; fetch models lists local models
- Jul 18Now supports the Gemma 4 QAT build