Back to repos

FreeToken

Python

Run frontier AI models on your gaming PC

FlashML-org/FreeToken on GitHub

What you get

Run 290B+ Mixture-of-Experts models like DeepSeek-V4-Flash and Qwen3.6-35B locally on your RTX 40/50 GPU in seconds. FreeToken elastically distributes the workload across GPU memory, CPU, and host RAM with bandwidth-adaptive execution. Desktop app for setup and tuning, CLI and OpenAI-compatible API for coding agents,fully offline, no API keys or cloud.

Why it exists

Bandwidth-adaptive co-execution: the $ policy balances GPU and CPU to maximize throughput on whatever hardware you have.

How it works

Semantic-aware caching remembers parts of context (tool calls, thinking blocks) across agent turns, avoiding redundant recomputation.

For
Developers who want to run state-of-the-art open models locally on their own hardware without cloud API costs or internet dependency.

Stars14k
Forks1.4k
Stars / day115.0
LicenceApache-2.0

Useful? Star it. That is the only thanks most maintainers get.

Also from FlashML-org