LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents

Details

Training coding agents with RL is complicated by the gap between training infrastructure and the native tool-use environments where agents run. LEGO-RL bridges this gap by proxying LLM calls in-process to capture raw token streams, enabling token-level log-probability alignment even when harnesses compress or re-serialize outputs. Three pillars make this work: faithful optimization (in-process proxying for accurate log-probs maintaining >0.99 rollout-training correlation), reliable execution (sandbox orchestration with image caching and reward hacking defenses), and observable training (automated validation with a Live UI for trajectory inspection). Critically, the framework does not modify any harness's internal control flow. Results using GSPO on Qwen3.5-35B-A3B — a sparse MoE model — are concrete: SWE-bench Verified improves across OpenHands SDK (64.0%→70.4%), Claude Code (62.4%→68.2%), and OpenCode (57.2%→66.6%). The rollout-training probability correlation stays above 0.99, validating the fidelity of the optimization. This is practically significant for teams building production coding agents. RL can meaningfully lift SWE-bench scores across multiple harnesses simultaneously, and the framework design is harness-agnostic.