SYLIN

SGLang

by SGLang Project · Runtime · Text & code

promising

Fast model-serving engine with measured gains; 1.0 just announced.

SGLang announced 1.0 at the SGLang Summit

AccessOpen codePricefree; open sourceRuns onmacOS, LinuxLicenseApache 2.0

Why we say so

SemiAnalysis and the SGLang team measured up to 10x gains on Rubin and 3x faster small-batch decoding on a TPU v7 host. Fahd Mirza used it to serve Qwen3.8-27B with two speculative drafters. The 1.0 announcement is the maker's own word, and part of the performance data comes from the project itself.

Evidence

  1. Nov 14
    Up to 10x Rubin gains in under 5 days; SGLang-TorchTPU serves Muse Spark Open on one TPU v7 host with 3x faster small-batch decode
  2. Sep 7
    Used to serve MiniCPM5-2B locally on one GPU
  3. Sep 1
    Reported to crash on mixed-GPU setups and pick wrong kernels, like vLLM, while offering paged attention and high concurrency
  4. Aug 26
    DFlash 2 support is said to be in SGLang, and the presenter believes it is also in vLLM and llama.cpp (unverified)

Alternatives

llama.cppPraised for one-binary deployment, heterogeneous hardware and wide quantization support, but called slow for concurrent use
Nari Qwen3-TTSAuthors say vLLM-Omni and SGLang-Omni are too slow for production and can break realtime playback at low latency