SYLIN

Ollama

Runtime · Text & code

promising

A usable local backend, but small prompt changes can wipe its prefix cache.

Found that one system-prompt timestamp or a change in reasoning effort can wipe the prefix cache

Why we say so

Simon Willison names Ollama as a local OpenAI-compatible backend that works with datasette-llm. Ollama's own testing found that a single system-prompt timestamp, or a change in reasoning effort, can wipe the prefix cache and slow repeated requests. There are no broader reviews here.

Evidence

  1. Nov 14
    Named as a local OpenAI-compatible backend usable with datasette-llm
  2. Sep 4
    Used via its API with Qwen 3.6 27B (needs tool use) on an RTX A6000 48 GB to summarize Obscura output
  3. Aug 26
    Works as a custom provider in DSH via Ollama's default endpoint; fetch models lists local models
  4. Jul 18
    Now supports the Gemma 4 QAT build

Runs on Ollama

31 models · newest first
Show all 31

Alternatives

llama.cppReviewer prefers plain llama.cpp over the Ollama and LM Studio wrappers; lower memory use once context is reduced