Qwen3.8-4B-Distill
by Empero AI · Model · Language & code models
Empero distilled the 2.4T-parameter Qwen teacher into a 4B student using ~45,000 curated reasoning traces; GGUF quants published
Evidence
- Aug 22Student is a 4B hybrid model with gated Delta net layers, trained to reproduce teacher reasoning traces
- Aug 22MMLU rises nearly 20% over the base 4B model; GSM8K math scores drop slightly, per the model card
- Aug 22On one ethical dilemma prompt all three quants reached the same decision; Q8 reasoned deepest (587 vs 311 words for Q4) with a more original framing