Bounded Semantic Planning and Deterministic Compilation for Reliable Enterprise Text-to-SQL
| Source: arXiv AI
Tags: text-to-SQL, enterprise AI, semantic parsing, NLP, LLM, insurance AI
Semantic Path Compilation (SPC) achieves 97.4% accuracy on an insurance enterprise SQL benchmark versus 55.3% for direct LLM-to-SQL, by separating stochastic intent interpretation from deterministic query construction.
Details
Researcher Yi Ai proposes a significant architectural shift in enterprise text-to-SQL: rather than asking a language model to simultaneously interpret a business question and write a complete SQL query, the work separates these responsibilities into distinct, purposefully designed stages. SPC (Semantic Path Compilation) uses a multi-turn planner that grounds natural language phrases into governed schema concepts, then hands off to deterministic code for graph traversal, role predicate selection, grain specification, and SQL construction. Only the natural language interpretation step is stochastic; the actual query assembly is code. On the ACME insurance benchmark (38 questions, 3 runs each), SPC was adjudicated correct on every run for 37 of 38 questions (97.4%), compared to 21 (55.3%) for a DDL-to-SQL baseline. The paired McNemar test shows p=3.05×10⁻⁵ — the performance difference is statistically robust. SPC produced zero adjudicated wrong-but-executed answers; the baseline produced 29. The paper is careful to note this is an end-to-end systems result, not a proof that compilation alone drives the improvement — SPC also receives pre-governed semantic artifacts that the baseline does not. GPT-5.4 and Gemini-3.6-Flash showed similar robustness, though per-run artifacts were not preserved for formal analysis.