We’ve open-sourced R-CLI.
Not a stripped-down version. Not a demo. Not a reference implementation.
The actual coding harness we use at Backboard.
You can inspect it, fork it, modify it, point it at your own models, and run it on your own infrastructure.
We decided to release it because of something our benchmarking has made increasingly difficult to ignore:
The model is only part of what determines how well an AI system performs.
For the last several years, most of the conversation around AI performance has focused on models. A new model launches, benchmarks move, leaderboards change, and everyone asks the same question: which model is best?
But when we started systematically testing coding agents, we kept seeing something else.
The exact same model could produce dramatically different results depending on the system wrapped around it.
Context management matters. Tool selection matters. Agent behavior matters. How the system recovers from mistakes matters. How it decides what information to keep, discard, or retrieve matters.
The harness matters.
And we now have some fairly compelling evidence of just how much.
Same Model. Different Result.
On Terminal-Bench 2.1, R-CLI achieved 85.4% using Claude Opus 4.8 across the full submission shape of 445 trials.
Claude Code’s published result using the same model is 78.9%.
The underlying model did not change.
The system around it did.
That R-CLI result placed above all 17 published entries we were comparing against, including Claude Code and Codex.
Then we tried something more aggressive.
We ran R-CLI using Kimi K3, with a single attempt per task and no frontier model anywhere in the run.
It reached 91%.
Total model cost: $28.72.
That 91% result is not a judged Terminal-Bench submission, and we do not present it as one. But we have made the verifier logs public so anyone interested can inspect what happened.
The implication is more interesting than the leaderboard position.
If changing the harness while keeping the model constant can materially change performance, then evaluating AI systems primarily by their underlying model is increasingly incomplete.
What matters is the system.
Why Open Source It?
That raises an obvious question.
If we believe the harness is this important, why give it away?
Because R-CLI was never the part of Backboard we intended to protect.
We want the harness to become infrastructure people can inspect, improve, and trust.
Open sourcing it also makes our claims falsifiable.
You do not have to take our word for the benchmark results. You can read the implementation. You can run it yourself. You can replace the models. You can change the prompts. You can break things. You can improve things.
That is much more interesting to us than publishing another benchmark screenshot.
R-CLI includes hundreds of skills, hundreds of MCP integrations, and the ability to work across a broad range of models and providers.
You can bring keys from OpenAI, Anthropic, OpenRouter, Grok, and others.
But the use case we find most interesting does not require sending your code to a frontier model provider at all.
You can point R-CLI at an open model running inside your own infrastructure.
That changes the equation.
Coding Agents Without Giving Up Your Code
Most of the best-known coding agents today depend on external model APIs.
For individual developers, that can be a perfectly reasonable trade.
For enterprises, governments, defence organizations, financial institutions, and companies working with sensitive intellectual property, it can become much more complicated.
Source code is often among the most valuable information an organization owns.
If using an advanced coding agent requires continuously transmitting that code outside the organization, there will always be environments where the answer is simply no.
Open-weight models are starting to create another path.
Their capabilities are improving quickly. The hardware required to run them continues to become more accessible. Quantization and inference techniques continue to improve what can be run on increasingly modest infrastructure.
What has been missing is the system around those models.
A capable model sitting on a GPU is not automatically a capable coding agent.
It still needs tools. Context. Planning. Memory. Execution. Recovery. Evaluation.
It needs a harness.
That is why we think open sourcing R-CLI matters.
It gives organizations a path to combine increasingly capable open models with a production-grade agentic system while retaining control over where their code runs.
No dependency on a single model provider.
No dependency on a single API.
No requirement that your source code leave your infrastructure.
And if a provider changes its pricing, policies, models, or roadmap, you can change what sits underneath the harness.
The Model Is Becoming a Component
We think the architecture of AI systems is starting to change.
Models will continue to improve, but access to capable models is becoming broader.
That means more differentiation will move into the layers surrounding them.
How efficiently can you use the model?
How well can you manage context?
How reliably can the agent complete long-running tasks?
Can you switch between models?
Can you run locally?
Can you operate in an air-gapped environment?
Can the system improve without forcing you to replace the entire stack?
These are increasingly infrastructure questions, not model questions.
R-CLI is our attempt to make one part of that infrastructure open.
We expect people to find things we got wrong. We expect people to improve it. We expect forks. We expect someone to run it with models and hardware combinations we have never tested.
That is the point.
The benchmark results got our attention.
What happens when everyone can use the harness is considerably more interesting.
R-CLI is open source today.
Read the code. Run the benchmarks. Point it at your own models.
And if you think we can make it better, send us a pull request.
Rob Imbeault
SHARE