SYLIN

DeepSeek V3 in JAX

by Vaibhaav Tiwari · Framework · Text & code

promising

Tiny readable DeepSeek V3 in JAX for learning; not a usable model.

~190-line pure-JAX DeepSeek V3 with Multi-head Latent Attention, bias-balanced MoE (no auxiliary loss) and multi-token prediction; checkpoint in v1.0 release

AccessOpen codeLicenseMIT

Why we say so

The author describes about 190 lines of JAX with a 36.2M-parameter model trained on 207 detective novels in 2.6 hours on a MacBook Air. An HN commenter ran it end to end on an AMD RX 7600 XT, but needed two workarounds to avoid a GPU hang. It is a teaching-scale project, and the only quality claims come from the author.

Evidence

  1. Nov 14
    Ran end-to-end on an RX 7600 XT 16GB under Fedora with ROCm; needed HSA_OVERRIDE_GFX_VERSION=11.0.0 and --xla_gpu_enable_triton_gemm=false to avoid a GPU hang