RecSA

by Hao AI Lab · Technique · Text & code

Sparse attention searching a tree over the KV cache; selection cost and entries read per query grow as O(log L)

Promised Nov 17: applying RecSA to larger models next (no date)

Evidence

  1. Nov 17
    Code released with kernels, Qwen3 integration, RULER evaluation and latency benchmarks
  2. Nov 17
    Initialized from Qwen3-4B and trained on 0.62B tokens at 32K, scores 60.5 on RULER at 1M versus 40.0 for the best baseline (Dense + DCA)
  3. Nov 17
    On one H200, prefill is 3.2x and decoding 3.8x faster than dense FlashAttention at 524K; decoding stays near 6 ms from 131K to 524K