Transformers Model Integration Contracts
| Source: Hugging Face Blog
Tags: Hugging Face, Transformers, transformers-mic, UMT5, PyTorch SDPA, Model Integration Contracts
Hugging Face proposed Model Integration Contracts for Transformers, a standard that packages shrunk model fixtures and checks to catch regressions on cheap CPU runners, after an SDPA change let UMT5's decoder see future tokens for nearly two months.
Details
On July 30, 2026, enabling PyTorch SDPA for T5 in Transformers caused UMT5's decoder to attend to future tokens during training. Nothing crashed and the bug stayed in main for almost two months; replaying those commits later, a small UMT5 fixture on CPU flagged it every time. Transformers supports 750+ model types, with 88,000+ CPU tests on main and 200,000+ GPU tests daily, so full integration tests cannot run on every PR. The team's answer is a fixture: a small model built from the real checkpoint's config with fewer, narrower layers but every layer type, and unchanged tokenizer and processor files. The UMT5 fixture used 66 MB versus a 1,227 MB source checkpoint. transformers-mic proposes packaging this as a contract with a pinned fixture, a runnable usage path and checks beyond a single forward pass.