SYLIN

Instella-MoE

by AMD · Model · Language & code models

16B MoE (2.8B active) trained from scratch on AMD Instinct GPUs with ROCm; checkpoints, recipes and code published

AccessOpen weightsNeeds~32 GBDownload32 GBSize16B parametersLicenseResearchRAIL

Evidence

  1. Aug 2
    Base model scores about 76.7 average, ahead of Moonlight 16B and Olmo 3 7B at fewer active parameters; post-trained is top fully open model on AMD's chart
  2. Jul 28
    AMD's first MoE language model: 16B total, 2.8B active parameters, trained end-to-end on Instinct MI300X/MI325X GPUs
  3. Jul 28
    Reviewer calls the benchmarks unremarkable, noting comparisons are against older or lesser-known models; none independently verified

Cost to run