SYLIN

NVFP4 training

by Thinking Machines Lab · Technique · Text & code

unproven

A research recipe for training in 4-bit; only the maker's claims so far

Native 4-bit forward pass from first pretraining step through final RL rollout; mixed-precision recipe detailed

StatusResearch

Why we say so

Thinking Machines says it ran the forward pass in 4-bit from the first pretraining step through the final RL rollout. The only other source, @teortaxesTex, repeats speed and memory figures (3.6x fewer bits than BF16, up to 6x faster on GB300) without independent testing. No one outside the lab has reproduced the results or reported on quality.

Evidence

  1. Nov 14
    NVFP4 stores weights in 3.6x fewer bits than BF16 and runs up to 6x faster on GB300

Alternatives

Works with