SYLIN

Gated MLA

by AMD · Technique · Text & code

Learned gate after multi-head latent attention that turns down weak signals, credited for strong results with few active parameters