Research · High impact2 sources
Why Your Coding Agent Costs 5x More Than It Should (It Is Not the Model)
DEV · By Felipe Castillo ·
The Brief · written from 2 sources
- Runta's FrontierHarness Eval v1.1 tested 22 coding agent harness configurations across 5 models, running 27,280 agent runs over 62 terminal tasks.
- The benchmark totals 83 billion tokens and finds the same model can cost up to five times more depending on the harness it is paired with.
- The benchmark was run by Runta and Baseten and covers open-weight models.
Why it matters
Choice of agent harness, not only the model, can drive running costs of coding agents substantially, so teams should measure cost per task when selecting tooling.
Who’s affected
- Teams running coding agents on open-weight models
- Developers choosing a coding agent harness
- FrontierHarness Eval v1.1
- #coding-agents
- #benchmark
- #llm-costs
- #agent-harness
- #open-weight-models
Runta published FrontierHarness Eval v1.1 today: 27,280 agent runs, 83 billion tokens, 22 harness configurations, 5 models, 62 terminal tasks. It is the largest public comparison of coding agents I have seen, and it is free to read. I read it expecting one answer and found…