This could be the largest synthetic code dataset yet

Details

IBM Research has released CodeAlchemy, a synthetic code data pipeline built to solve an increasingly real problem: the best publicly available code on the web is either locked behind licenses or has already been absorbed into existing models. The pipeline covers 15 programming languages and has generated nearly 1 trillion tokens — at least 200× larger than Wikipedia. What makes CodeAlchemy structurally different from prior datasets is the 1.3 million code files paired with their actual execution traces. Most code LLMs learn from static code, which teaches the model what code looks like — not what it does. By including line-by-line runtime traces, CodeAlchemy gives models a signal about how variables change, how branches resolve, and where programs fail — the kind of knowledge a developer builds through manual debugging. The pipeline uses LLM-assisted rewriting to elevate low-quality web code — a strategy IBM partly credits to the technique used by the Kimi K2 team at Moonshot AI. The entire pipeline, not just the final dataset, is being open-sourced, meaning any lab can replicate or extend the generation approach. IBM built CodeAlchemy to improve its own Granite family of code models. Independent benchmarks of the full dataset's downstream impact on third-party models have not yet been published, so quality claims are currently IBM's own.