JetSpec
by Hao AI Lab · Technique · Text & code
Lossless speculative decoding using causal tree drafting, claimed up to 9x faster generation with unchanged output
Evidence
- Sep 1Shown finishing the same answer faster than DFlash and plain autoregressive decoding in the project's demo
- Sep 1Each branch of the draft tree attends only to its own earlier guesses, so more of the tree survives verification
- Sep 1On an H100 with a Qwen 8B model, baseline 28.03 tok/s rose to a 2.82x speedup, below the claimed 9x