Meet billion-context
Why context windows fill up
Every turn of an AI coding session resends the whole conversation back to the model. Tool outputs, logs, file reads and retries pile up far faster than anything you typed. By the time a task gets interesting, a session can burn through a 200K window in an afternoon — after which quality degrades or the session dies.
What sits between your agent and the model
billion-context runs locally, loopback-only, between your coding agent and the model provider. It keeps a token-level ledger for every request and compresses the older parts of the conversation into dense, referenceable summaries while the most recent context stays byte-exact. Cadence is growth-driven (roughly 50K tokens between folds), not percentage-driven.
Two modes cover both integration styles: plugin mode, where the agent owns compression (for example the pi extension), and proxy mode, where the proxy performs it server-side and plain clients never notice.
What you get
- Up to 5x fewer cumulative input tokens
- A single session measured across a month of real development
- An attribution panel: cache hit rate, net savings, active blocks — per request, in the built-in web UI
The numbers on the front page are production data: 174,327 model calls, 18.8B cumulative input tokens, zero window violations over 4.5 months on three hosts.
Where it runs
One npm package, Node 22+, MIT license. Install with npm install -g billion-context --prefix=~/.local (on nvm / Homebrew Node or Windows, drop --prefix), start the proxy with bili, and open <http://127.0.0.1:8787/__bili/>. Full configuration lives at ~/.config/billion-context/billion-context.json — { } is a valid configuration. Read the docs for everything else.
Note: this is a sample post demonstrating the blog format (a markdown subset rendered to static pages by
tools/build-blog.mjs). The first real story will replace it.