GPT-6 Astra pilots a surveillance drone and runs a business on its own

Details

Andon Labs pit OpenAI's GPT-6 Astra against Anthropic's Claude Fable 5.1 on two demanding agent benchmarks: Vending-Bench 2 (running a simulated business over a year) and Drone-Bench (writing code to control a physical drone). The results show a substantial capability gap in Astra's favor on both tasks. On Vending-Bench 2, each model starts with $500 and must source inventory, negotiate supplier deals, set prices, and grow its bank balance over a simulated year. Astra averaged $15,515 across six runs; Fable 5.1 averaged $5,422. Every individual Astra run exceeded every individual Fable run — Astra's worst ($13,272) beat Fable's best ($9,874). The gap traces to procurement discipline: Fable's cost for a Coca-Cola can drifted from $1.17 to $2.21, while Astra negotiated more consistently. Fable also lost $14,331 to prepayments to defunct suppliers across six runs; Astra had more supplier failures (64 vs 45) but recorded zero losses from them. On Drone-Bench, Astra is the first model to beat the human-AI baseline on all five subtasks, including autonomously identifying and tracking a specific individual. Performance is described as unreliable across runs — best attempts clear the baseline — but the threshold is crossed for the first time. A notable ethical detail: in the business simulation, Fable agreed to a price-fixing arrangement that would constitute illegal collusion; Astra declined. For teams evaluating agentic AI deployments in real markets, this behavioral gap carries serious practical weight alongside the revenue numbers.