LAVA: Logic-Aware Validation and Augmentation Framework for Large-Scale Financial Document Auditing
| Source: arXiv AI
Tags: financial-AI, document-processing, LLM, multimodal, enterprise-AI, hallucination
LAVA, a modular multimodal LLM pipeline for financial document validation, outperforms baselines on hallucination control and edge-case handling across real-world payroll, tax, and loan underwriting documents — with traceable symbolic verification of arithmetic rules.
Details
Financial document validation in production — payroll auditing, tax compliance, loan underwriting — demands accuracy and reproducibility that current LLM pipelines struggle to deliver. Documents arrive with heterogeneous layouts, context-dependent content, and embedded arithmetic business rules that LLMs hallucinate against. LAVA addresses this with a four-stage modular design built on multimodal LLMs. The four stages are: document-rule retrieval (matching the applicable validation rules to document context), layout-preserving information extraction (parsing heterogeneous formats without losing table structure or positional context), auxiliary metadata enrichment (supplementing with related data or calculation inputs), and symbolic/arithmetic verification (applying exact computational checks on extracted values rather than trusting LLM arithmetic). The symbolic verification stage is the critical differentiator — LLMs reliably make arithmetic errors on financial figures. Evaluated on a large real-world benchmark with diverse financial documents and dozens of expert-curated validation rules, LAVA outperforms baselines in hallucination control and edge-case handling while maintaining efficient token usage. Fine-grained error attribution lets auditors trace exactly which stage and rule triggered any validation failure. The paper was published at FinNLP 2025 at ACL.