Study: Fancy coding agent harnesses raise costs without improving outcomes
In brief
- Advanced coding agent harnesses do not significantly improve pass rates.
- Task cost can triple due to repeated system prompt transmission.
- Rerunning the same setup remains as effective as switching interfaces.
A controlled study finds that switching between coding-agent harnesses like Claude Code, OpenCode or mini-SWE-agent barely improves pass rates compared to rerunning the same setup. However, the cost per task can triple, largely due to the system prompt being resent at every step according to the report.
This summary gives you the essentials. The full article, its figures and quotes are on The Decoder.
Read the original articleAI-generated summary of the source, checked by rules. How we write · Report an error