Fidelity Collapses with Table Size, Not Turn Count: Delivery Architecture in a Deployed Tool-Using Agent

| Source: arXiv AI

Tags: tool-using agents, GPT-4o-mini, Gemini 2.5 Pro, EMNLP 2026, REALM workshop, agent architecture

In a deployed award-travel agent, keeping tool records out of the model context raised fully correct turns to 92.3% versus 45.7%, with in-context answers collapsing once tables exceed about 74 rows.

Details

The study compares SYNTHESIS, which feeds full record sets into the planner context, with BYPASS, the deployed design that keeps records in session state, turns follow-ups into filter arguments and delivers rows over a typed side channel. Across 24 dialogues and 339 scored turns each, BYPASS was fully correct on 92.3% of turns and SYNTHESIS on 45.7%. SYNTHESIS is near-perfect at 10 rows or fewer and drops to 8% or less beyond about 74 rows. The threshold scales with model capability, from under 10 rows for GPT-4o-mini to about 198 for Gemini 2.5 Pro, while BYPASS stays flat. Beyond 100 rows each correct SYNTHESIS answer costs about 200x BYPASS's $0.012.