Failure-Driven Workflow Refinement

| Source: arXiv AI

Tags: CE-Graph, LLM Workflows, Agentic Workflow Optimization, Counterexample-Guided Refinement, Expected Failure Mass

CE-Graph refines LLM workflows by modeling where they fail rather than optimizing a single score, editing workflow graphs to reduce the 'expected failure mass' found in a pool of counterexamples.

Details

Workflow optimization for LLM systems usually scores candidate workflows with one scalar metric. The authors argue this collapses rich execution traces into pass/fail signals and hides the structure of failures. They reframe the goal as minimizing Expected Failure Mass, the integral of failure probability over a Failure Signature Space. CE-Graph approximates the failure distribution from counterexamples, finds the densest regions as recurring failure modes, and applies operator-constrained graph edits through a Propose-and-Verify loop. The paper reports evaluations on math, code and QA benchmarks. This is a revised version (v2) of a paper first submitted in October 2025.