DualWorldBench: Can Agents Plan Deliveries across Symbolic and Grounded Worlds?

| Source: arXiv AI

Tags: DualWorldBench, agent benchmarks, multimodal agents, embodied planning, TextWorld

DualWorldBench tests agents on pickup-and-delivery planning in symbolic and grounded worlds and finds they understand local scenes but struggle to build globally feasible multi-goal plans.

Details

Existing agent evaluations often use simple tasks, leaving unclear whether agents can combine perception and reasoning for multi-goal planning under temporal, physical and multimodal constraints. DualWorldBench uses a shared pickup-delivery interface across three parts: TextWorld for symbolic planning with varying task sets and release timing, SimWorld for grounded path planning with text, image or multimodal input, and DualWorld-VQA for perception and planning diagnostics. Agents struggled to find optimal task orders under interacting time constraints and did not reliably turn grounded observations into feasible actions. Good perception did not translate into constrained planning. Data and code are public.