SGLang
by SGLang Project · Runtime · Text & code
promising
Fast model-serving engine with measured gains; 1.0 just announced.
SGLang announced 1.0 at the SGLang Summit
Why we say so
SemiAnalysis and the SGLang team measured up to 10x gains on Rubin and 3x faster small-batch decoding on a TPU v7 host. Fahd Mirza used it to serve Qwen3.8-27B with two speculative drafters. The 1.0 announcement is the maker's own word, and part of the performance data comes from the project itself.
Evidence
- Nov 14Up to 10x Rubin gains in under 5 days; SGLang-TorchTPU serves Muse Spark Open on one TPU v7 host with 3x faster small-batch decode
- Sep 7Used to serve MiniCPM5-2B locally on one GPU
- Sep 1Reported to crash on mixed-GPU setups and pick wrong kernels, like vLLM, while offering paged attention and high concurrency
- Aug 26DFlash 2 support is said to be in SGLang, and the presenter believes it is also in vLLM and llama.cpp (unverified)