Further Notes on Our Recent Research on AI Delegation and Long-Horizon Reliability

Details

Microsoft Research published a clarifying post on their paper 'LLMs Corrupt Your Documents When You Delegate,' which sparked discussion about AI reliability in long-horizon agentic workflows. The paper introduces DELEGATE-52, a benchmark that measures information preservation when AI systems handle repeated modifications to documents, spreadsheets, code, or structured files without human verification between steps. The core finding: frontier models accumulate semantic fidelity degradation over repeated edits. Across evaluated settings, models showed roughly 19–34% degradation in artifact fidelity over 20 delegated iterations — sparse but consequential errors that compound over time. Python workflows were a notable exception, showing less than 1% average degradation, likely because Python's structural constraints help models preserve semantic meaning more reliably. Microsoft is careful to frame DELEGATE-52 as a diagnostic stress test, not a verdict against AI in professional workflows. The benchmark measures semantic content preservation — not task success or user satisfaction. A model can complete a task while silently corrupting key information, a distinction that matters enormously in document-heavy enterprise use cases. Mitigation strategies already exist: verification loops, orchestration layers, and domain-specific tooling can counteract degradation in production systems. The research is part of Microsoft's ongoing effort to close the gap between benchmark performance and real-world long-horizon task reliability.