Walk Before You Run: The Importance of Data Exploration for Data Analysis Agents
| Source: arXiv AI
Tags: LLM agents, data analysis, spreadsheets, benchmarks, VLDB 2026, human-in-the-loop
Accepted at VLDB 2026 workshop, this paper shows that LLM-based data analysis agents systematically miss logical table structure in multi-sheet spreadsheets even when reading the content — and that an explicit Data Exploration stage, treating schema recovery as a first-class inspectable artifact, consistently improves downstream task correctness.
Details
LLM-based data analysis tools — from spreadsheet Q&A to code generation over uploaded files — are evaluated almost exclusively by the accuracy of their final answers. This paper argues that reliable analysis requires a prior step that current tools and benchmarks leave implicit: understanding what the dataset actually contains.\n\nFor complex workbooks, Data Exploration means identifying logical tables behind physical sheets, interpreting column semantics, recovering keys and relationships, and detecting data quality issues. The authors show that skipping this step creates a systematic gap between downstream task performance and the dataset understanding actually needed for reliable, human-checkable analysis.\n\nTwo benchmarks formalize the evaluation: a real multi-sheet workbook benchmark built from a Vitamin D study dataset, and an extension of DSBench with schema-fixed Data Exploration artifacts. Systems are scored on the quality of a structured artifact capturing tables, columns, semantic roles, relationships, and profiling signals.\n\nKey finding: strong LLMs and data-analysis agents still miss important logical structure even when reading spreadsheet content. Explicit Data Exploration support improves downstream correctness and provides a natural human-in-the-loop checkpoint where domain experts can review and correct the artifact before downstream analysis proceeds. Accepted to the VLDB 2026 DASHSys workshop.