Catalyzing Interactive HTML Generation Capabilities via High-Fidelity Reinforcement Learning

| Source: arXiv AI

Tags: WebCatalyst, HTML generation, reinforcement learning, Qwen3.6-35B-A3B, WebGen-Bench, reward hacking

WebCatalyst trains models to generate interactive HTML by playing their own outputs in a browser, lifting Qwen3.6-35B-A3B on Cookie-Bench Mini Games from 68.8 to 75.3 and ahead of Claude Opus 4.7 in average rank.

Details

Code LLMs often produce HTML pages that render but break during real interaction. Using mini-games as a demanding testbed, the authors show weaknesses in interaction experience, reward signal and credit assignment. WebCatalyst is a browser-interaction RL framework with three parts: a game-level GUI assessor that plays and evaluates outputs, Visual Anchor to stop vision-language reward hacking, and Action-Trace Credit to tie token-level credit to runtime execution traces. Trained on fewer than 1K requests, mostly games, it raised Qwen3.6-35B-A3B from 68.8 to 75.3 on Cookie-Bench Mini Games and from 48.8 to 56.9 on WebGen-Bench, beating Claude Opus 4.7 in average rank across metrics. The authors conclude that feedback fidelity is the main lever for better interactive HTML generation.