PersonaCUABench: Evaluating computer-use agents based on user personas
In brief
- PersonaCUABench evaluates computer-use agents by changing only the user's profile.
- Task success drops by 36% when requests are rewritten by personas.
- Only 62.7% of tasks fully satisfy the user, versus 84.1% for original requests.
A new benchmark, PersonaCUABench, tests computer-use agents by varying only the user's profile who gives the instruction. Results show a 36% drop in task success compared to original requests, highlighting the need to assess real user needs satisfaction.
This summary gives you the essentials. The full article, its figures and quotes are on cs.AI updates on arXiv.org.
Read the original articleAI-generated summary of the source, checked by rules. Report an error