Speculative Self-Distillation
Technique · Text & code
Teacher intervenes only when student drifts; matches on-policy accuracy with 38-69% fewer tokens and less forgetting
StatusResearch
Technique · Text & code
Teacher intervenes only when student drifts; matches on-policy accuracy with 38-69% fewer tokens and less forgetting