Evaluating deep search agents on real-world web environments
In brief
- New OpenDS-World benchmark evaluates autonomous web search agents in real-world settings.
- 200 complex tasks with human-verified answers and evidence across formats.
- Top model hits 39 % accuracy, humans 21.1 %.
Researchers introduce the OpenDS-World benchmark to assess agents capable of performing deep search in live, open-web environments. According to the announcement, the benchmark includes 200 complex tasks across diverse data formats and modalities, with human verification to validate answers.
Results show no model reaches 40 % accuracy, with a clear gap between model performance (39.0 % at best) and human performance (21.1 %). Analyses highlight persistent challenges in tool coordination and long-horizon search management.
This summary gives you the essentials. The full article, its figures and quotes are on Hugging Face Daily Papers.
Read the original articleAI-generated summary of the source, checked by rules. How we write · Report an error