Import AI 479: Spacewar; low-latency tasks versus LLMs; Genie 4

| Source: Import AI (Jack Clark)

Tags: Import AI, Jack Clark, Realtime-Bench, NVIDIA, AI agents, Genie 4

Import AI 479 highlights Realtime-Bench, a new benchmark showing current LLMs perform poorly on tasks where delayed actions hurt success, such as games and online purchases.

Details

Jack Clark's Import AI 479 covers Realtime-Bench, a suite of 50 interactive tasks: 30 web tasks across travel, e-commerce, service and security, and 20 game tasks across arcade, puzzle, strategy and simulation. Built by researchers from UT Austin, Ohio State, UIUC, NVIDIA, Northwestern and Steadyworks, it tests three abilities: tracking a changing world (RT-Perceive), adjusting reasoning depth to available time (RT-Deliberate), and timing actions well (RT-Act). Tasks run under wall-clock, paused-clock and token-clock conditions, separating raw capability from latency effects. The issue also covers Spacewar and Genie 4. For teams building agents for auctions, ticketing or live interfaces, latency remains a real constraint.