SYLIN

Qwen3.8-4B-Distill

by Empero AI · Model · Language & code models

Empero distilled the 2.4T-parameter Qwen teacher into a 4B student using ~45,000 curated reasoning traces; GGUF quants published

AccessOpen weightsNeeds~2.8 GBDownload2.8 GBSize4B parametersLicenseApache 2.0

Evidence

  1. Aug 22
    Student is a 4B hybrid model with gated Delta net layers, trained to reproduce teacher reasoning traces
  2. Aug 22
    MMLU rises nearly 20% over the base 4B model; GSM8K math scores drop slightly, per the model card
  3. Aug 22
    On one ethical dilemma prompt all three quants reached the same decision; Q8 reasoned deepest (587 vs 311 words for Q4) with a more original framing

Cost to run

Alternatives

Made from