GlobeBench: Benchmarking Text-Based World Models for Scalable Simulations
In brief
- GlobeBench tests 1,632 tasks across five domains to evaluate text-based world models.
- Performance drops to 5.1% when actions impact multiple dependent objects.
- 631 tasks remain unsolved by all models, exposing a major capability gap.
GlobeBench introduces a benchmark of 1,632 tasks to assess text-based world models' ability to simulate complex environments. Results highlight critical limitations, with performance dropping sharply when actions affect multiple dependent objects. No model solves 631 tasks, revealing a significant gap in general LLM capabilities.
This summary gives you the essentials. The full article, its figures and quotes are on cs.AI updates on arXiv.org.
Read the original articleAI-generated summary of the source, checked by rules. Report an error