SYLIN

TokenWing

Runtime · Text & code

Developer claims it streams a majority of routed MoE experts from SSD; a user reported 45-50 tok/s on a 120B MoE with 12 GB VRAM and 32 GB RAM