SYLIN

Colibri

by JustVugg · Runtime · Text & code

Hy3 port adds a GPU tier: hot experts pinned in VRAM, warm in RAM, cold streamed from disk, and promoted by use

AccessOpen codePricefreeRuns onWindows, macOS, LinuxLicenseApache 2.0

Evidence

  1. Jul 22
    Expert cache hit rate rose from 21% cold to 71% warm, holding about 113 GB in RAM; speeds improve with use
  2. Jul 22
    Hy3 on an H100 with 40 GB VRAM budget ran slowly but faster than CPU-only; GPU use mostly stayed around 1-2 GB
  3. Jul 21
    Engine runs GLM 5.2 quantized to int4, keeping the multi-token-prediction head at int8 so speculative decoding keeps working

Alternatives

Works with