
colibri
Inference for 744B models on hardware you own

What you get
Run frontier mixture-of-experts models, 744B up to 2.8T parameters, on hardware you already own. Pure C, zero engine dependencies. It treats storage, RAM and VRAM as one inference hierarchy, so the model does not have to fit in your GPU to run on it. Eight model families work today including GLM-5.3, Kimi K3 and DeepSeek V4 Flash.
Why it exists
The interesting open models stopped fitting on normal machines. A 744B parameter model assumes a rack, so the choice for everyone else is renting somebody's GPUs or not running it at all.
How it works
AI memory multitiering: disk, system RAM and VRAM are treated as tiers of a single memory hierarchy rather than a hard boundary the weights must fit inside. The engine is pure C with no dependencies, which is what lets it run across consumer and mixed hardware instead of one blessed configuration.
For
Researchers and builders who want to run frontier models privately on their own hardware without GPU requirements or API costs.
- Use cases
- / Run GLM-5.2 (744B) on a laptop with disk
- / Inference on multi-GPU workstations with partial expert residency
- / Local experimentation with large models
Useful? Star it. That is the only thanks most maintainers get.
Also from JustVugg
- colibriโ 38k