Back to repos

colibri

C

Inference for 744B models on hardware you own

JustVugg/colibri on GitHub

What you get

Run frontier mixture-of-experts models, 744B up to 2.8T parameters, on hardware you already own. Pure C, zero engine dependencies. It treats storage, RAM and VRAM as one inference hierarchy, so the model does not have to fit in your GPU to run on it. Eight model families work today including GLM-5.3, Kimi K3 and DeepSeek V4 Flash.

Why it exists

The interesting open models stopped fitting on normal machines. A 744B parameter model assumes a rack, so the choice for everyone else is renting somebody's GPUs or not running it at all.

How it works

AI memory multitiering: disk, system RAM and VRAM are treated as tiers of a single memory hierarchy rather than a hard boundary the weights must fit inside. The engine is pure C with no dependencies, which is what lets it run across consumer and mixed hardware instead of one blessed configuration.

For
Researchers and builders who want to run frontier models privately on their own hardware without GPU requirements or API costs.

Stars38k
Forks4.1k
Stars / day270.3
LicenceApache-2.0

Useful? Star it. That is the only thanks most maintainers get.

Also from JustVugg