FreeToken
by FlashML · Runtime · Text & code
Edge-native MoE serving engine keeps most experts in system RAM and moves only needed experts to the GPU, so huge MoE models run on one consumer GPU.
Evidence
- Aug 25Built-in benchmark measures CPU compute versus PCIe transfer per quantization format and auto-picks hybrid or offload mode; NVFP4 chose offload on an A6000.