Deletion Tuning
by Agentic Learning AI Lab · Technique · Text & code
unproven
Research technique with big claims from its authors; nobody else has tested it yet
Learns a binary mask zeroing under 0.05% of gate-projection weights; claims to match full fine-tuning with masks hundreds to thousands of times smaller and less forgetting
StatusResearch
Why we say so
The Agentic Learning AI Lab and Lu et al. (apparently the same work) say a tiny mask, zeroing under 0.05% of weights, matches full fine-tuning with less forgetting. All claims come from the authors, with no independent replication or hands-on reports. Code and masks are public, so it can be checked.
Evidence
- Nov 11Learns a binary mask zeroing under 0.05% of gate-projection weights, matching full fine-tuning on RLVR, SFT and distillation with far smaller checkpoints
- Nov 11In RL, prior-task accuracy stays near base while full fine-tuning, LoRA and random sparse tuning lose up to 5.4 points