When Context Misleads: Intent-Guided Decoding for Robust Retrieval-Augmented Generation

Details

Researchers from the University of Sydney have proposed Intent-Guided Decoding (IGD), a decoding-time framework that addresses the trust-calibration problem in Retrieval-Augmented Generation. Standard RAG systems apply a fixed policy toward retrieved context: always trust it, or always prefer parametric memory. IGD instead infers user intent — does the user want the model to follow the retrieved context faithfully, or to correct it when wrong? — and adjusts decoding accordingly. The mechanism operates at two levels: answer-level filtering removes obviously misleading context before generation, and token-level correction steers individual token probabilities between the retrieved context and the model's parametric knowledge during the decoding pass itself. Evaluated across three faithful QA benchmarks and three factual-conflict benchmarks using five different LLMs, IGD improves factual recovery by up to 65.4 percentage points on factual-conflict scenarios versus direct RAG, while maintaining or improving context-following behavior on faithful benchmarks. For practitioners building RAG pipelines, this is directly applicable: a decoding-time intervention that does not require retraining. The approach's reliance on correctly inferring user intent at runtime is its main practical uncertainty.