Extra test-time compute doesn't always add evidence in LLM reasoning
In brief
- LLM errors are correlated even with extra compute, undermining test-time scaling methods' core assumption.
- DIA detects reasoning instability and consults external models to arbitrate, without reusing initial predictions.
- On 7 benchmarks, DIA improves performance in 12 out of 14 cases, with up to 7% gain on MuSiQue.
Research reveals that LLM errors remain correlated even with increased computation, challenging the assumption that more compute equals more independent evidence. The authors introduce DIA, a method that flags reasoning instability and consults external models to arbitrate, boosting performance across multiple benchmarks.
This summary gives you the essentials. The full article, its figures and quotes are on cs.AI updates on arXiv.org.
Read the original articleAI-generated summary of the source, checked by rules. Report an error