Instella-MoE
by AMD · Model · Language & code models
16B MoE (2.8B active) trained from scratch on AMD Instinct GPUs with ROCm; checkpoints, recipes and code published
AccessOpen weightsNeeds~32 GBDownload32 GBSize16B parametersLicenseResearchRAIL
Evidence
- Aug 2Base model scores about 76.7 average, ahead of Moonlight 16B and Olmo 3 7B at fewer active parameters; post-trained is top fully open model on AMD's chart
- Jul 28AMD's first MoE language model: 16B total, 2.8B active parameters, trained end-to-end on Instinct MI300X/MI325X GPUs
- Jul 28Reviewer calls the benchmarks unremarkable, noting comparisons are against older or lesser-known models; none independently verified