SYLIN

Deletion Tuning

by Agentic Learning AI Lab · Technique · Text & code

unproven

Research technique with big claims from its authors; nobody else has tested it yet

Learns a binary mask zeroing under 0.05% of gate-projection weights; claims to match full fine-tuning with masks hundreds to thousands of times smaller and less forgetting

StatusResearch

Why we say so

The Agentic Learning AI Lab and Lu et al. (apparently the same work) say a tiny mask, zeroing under 0.05% of weights, matches full fine-tuning with less forgetting. All claims come from the authors, with no independent replication or hands-on reports. Code and masks are public, so it can be checked.

Evidence

  1. Nov 11
    Learns a binary mask zeroing under 0.05% of gate-projection weights, matching full fine-tuning on RLVR, SFT and distillation with far smaller checkpoints
  2. Nov 11
    In RL, prior-task accuracy stays near base while full fine-tuning, LoRA and random sparse tuning lose up to 5.4 points