Cerebras Inference
by Cerebras · App or service · Text & code
unproven
Interesting hybrid GPU and Cerebras serving design, but only Cerebras's own claims so far.
Disaggregated inference in production: GPUs (AMD Instinct) do prefill, Cerebras CS-3 does decode, joined by KV Portal over RDMA
AccessAPI
Why we say so
Cerebras says the setup (AMD GPUs for prefill, CS-3 for decode) gives 5x tokens per megawatt with no loss in speed on GPT-OSS-120B, and that partners are in early access. No independent measurement exists. HN commenters welcomed the candor that the chips suit decode better, but want caching pricing, which the post does not cover.
Evidence
- Nov 18Claims 5x tokens per megawatt versus CS-3-only serving of a shared GPT-OSS-120B endpoint, with no loss in generation speed
- Nov 18Commenters welcome the admission that Cerebras chips suit decode better than prefill; the earlier blog post dodged the question
- Sep 3Added Qwen 3.8 27B to pay-as-you-go tier, replacing Gemma 4 31B there
- Sep 3Public endpoint rate limits (150k to 450k tokens/min, cached tokens counted) hit quickly in coding use
- Aug 19Cerebras Code offers GLM 4.7 only as a preview to be removed, leaving gpt-oss-120B; commenter says regular developers are neglected