DeepInfer
by DeepSeek · Runtime · Text & code
unproven
DeepSeek's serving engine is open, but its production economics can't be reproduced.
DeepSeek open-sourced DeepInfer, the inference engine it uses to serve its models at scale across thousands of GPUs
Why we say so
The performance figures (155 trillion tokens a day, about 80% margin) are DeepSeek's own claims. AI Search reports that the caching and load-balancing infrastructure was not released, so installing it won't match those economics. No one has reported using it hands-on yet.
Evidence
- Oct 23DeepSeek reports a cluster handling 155 trillion tokens per day, $38 revenue per GPU hour (about 80% estimated margin), 96% input cache hit rate