Import AI 466: The bitter lesson for robotics, AIs complete week-long programming tasks; and OpenAI’s accidental AI hacker

July 27, 2026 | Source: Import AI (Jack Clark)

Tags: MirrorCode, Epoch, METR, Claude Opus 4.7, GPT-5.5, Benchmarks, Long-horizon coding, AI agents

Epoch and METR's MirrorCode benchmark shows Claude Opus 4.7 completing a software reimplementation task estimated at 2-17 weeks of human effort in 14 hours for $251 — strong evidence that AI is crossing the threshold for autonomous long-horizon engineering work.

Details

MirrorCode, a new benchmark from Epoch and METR, tests AI systems on a task with no good shortcut: reimplementing an existing software program from scratch using only command-line access, without seeing the source code or searching the web. This design forces genuine architectural reasoning rather than code translation or memorization. The results are striking. Claude Opus 4.7 successfully reimplemented pkl — Apple's programmable configuration language at 61k lines of code — in 14 hours at $251 in inference costs. METR and Epoch estimate this task would take a human engineer 2-17 weeks. Both Opus 4.7 and GPT-5.5 completed gotree (16k lines) across multiple programming languages at costs between $100-400. Across 25 target programs, 17 had at least one perfect-scoring run, with 4 more reaching 99%+. Year-over-year progress is steep and worth noting: leading models from a year ago scored around 30% and were limited to simple programs like calendar utilities. Current frontier models are dramatically better. AI still struggles with the hardest cases — ruff (Python linter), giac_subset (mathematics package), and mailauth (email authentication library) remained unsolved — suggesting meaningful ceiling effects persist. Import AI #466 also covers Jack Clark's analysis of 'the bitter lesson for robotics' and an incident involving OpenAI's 'accidental AI hacker,' though the extracted content does not include full detail on those additional sections.