The Model's Tell: Measuring Context-Leakage Attack Signals with Behavior Gauges

Details

As LLMs are deployed with proprietary system prompts or RAG-retrieved documents, attackers can craft inputs that cause the model to leak that confidential context. Previous defenses required extracting hidden states — expensive and deployment-hostile. LeakGauge takes a different approach: probing the signal that leakage leaves in token probabilities before decoding, with no access to internals. The approach appends a suffix that gauges leakage behavior, then maps its prefill token probabilities to a risk score. A content-agnostic gauge (one that verbalizes leakage behavior rather than using the actual confidential content as the probe) turns out to be more robust than a direct gauge. Across 11 LLMs — including models as large as GLM-5.2 at 753B parameters and Kimi-K3 at 2.8T — LeakGauge achieves AUROC of 0.944 to 0.996 on unseen attack types. The signal remains stable when confidential content changes language or when attacks shift from verbatim to semantic disclosure. An activation-steering experiment links the observable risk score to an internal leakage-related representation direction. The resulting input detector uses fewer than 500 additional parameters and adds 10.34 ms of latency. For teams deploying LLMs with proprietary prompts or sensitive retrieved context, this is a practical, low-overhead detection layer.