SYLIN

FreeToken

by FlashML · Runtime · Text & code

Edge-native MoE serving engine keeps most experts in system RAM and moves only needed experts to the GPU, so huge MoE models run on one consumer GPU.

AccessOpen codeRuns onWindows, LinuxLicenseApache 2.0

Evidence

  1. Aug 25
    Built-in benchmark measures CPU compute versus PCIe transfer per quantization format and auto-picks hybrid or offload mode; NVFP4 chose offload on an A6000.