JHiMA-WebRL: A Joint Hierarchical Multi-Agent Reinforcement Learning Framework for Long-Horizon Web Tasks

| Source: arXiv AI

Tags: JHiMA-WebRL, WebArena, GRPO, web agents, multi-agent RL, ScaleCUA

JHiMA-WebRL jointly trains a 7B task decomposer and a 7B actor with RL and reaches 37.2% success on WebArena-Lite-v2, versus 24% for a 32B single-agent baseline trained on the same tasks.

Details

Long-horizon web tasks need planning, navigation, progress tracking and visually grounded actions, which strains single monolithic policies. Earlier multi-agent setups usually freeze the planner while training the actor. JHiMA-WebRL couples a task decomposer with an actor and trains both. A strong teacher model builds structured subtask supervision from multimodal GUI trajectories to fine-tune both roles, then an extension of GRPO jointly optimizes them in WebArena using hierarchical rollout grouping and role-specific advantages. On WebArena-Lite-v2, the 14B system (7B actor plus 7B decomposer) reached 37.2% average success, compared with 24% for a ScaleCUA-32B agent RL-tuned on the same tasks under the same protocol. Ablations show joint updates beat training the actor alone, suggesting role factorization with smaller models can compete with larger single agents.