DeliverBench: Benchmarking AI Agents on Delivering Realistic Multi-Stage Professional Projects

| Source: arXiv AI

Tags: DeliverBench, agent benchmarks, multi-stage projects, financial due diligence, legal drafting, arXiv

DeliverBench tests AI agents on 11 multi-stage professional projects; the best of seven systems scored only 65 of 100 on final delivery, showing task skill does not add up to project delivery.

Details

DeliverBench evaluates whether agents can carry a project from brief to final delivery rather than solving isolated tasks. Its 11 projects, covering financial due diligence, legal contract drafting and drug safety reporting, were built with 13 domain experts and chain 150 subtasks through 2 to 15 rounds of new materials and client feedback. No reference outputs are injected, so agents inherit their own earlier mistakes. Expert rubrics score each checkpoint and the final package. Across seven model-harness systems, the best scored 65.0 out of 100 on final delivery and passed 50.0% of subtasks at a 90-point threshold. In one audit project, all seven addressed review findings but only one preserved traceability.