Empty shelves or lost keys? Recall is the bottleneck for parametric factuality

August 12, 2026 | Source: Google Research Blog

Tags: Google Research, LLM factuality, WikiProfile, knowledge profiling, Gemini, GPT-5, chain-of-thought

Google Research's knowledge profiling framework finds frontier LLMs (Gemini 3, GPT-5) encode nearly all facts correctly but fail to recall many of them — most factual errors are retrieval failures, not knowledge gaps, shifting the optimal fix toward post-training and inference-time methods rather than more pretraining data.

Details

Google Research introduced knowledge profiling, a framework that separates two distinct failure modes: encoding (a fact was never learned) versus recall (a fact is encoded but inaccessible without external cues). Using WikiProfile — a benchmark of 2,150 Wikipedia-derived facts, each probed with ten questions covering encoding, recall, and recognition — researchers evaluated frontier LLMs including Gemini 3 and GPT-5. The core finding: frontier LLMs encode nearly all facts, yet recall many fewer of them. Standard accuracy metrics collapse these two failure modes, misleading practitioners who attribute errors to insufficient training data. Recognition (identifying the right answer among alternatives) succeeds where unprompted recall fails, confirming the facts are encoded but inaccessible. Practically, this reframes the intervention strategy. If encoding were the problem, larger models and more data would fix it. Since recall is the bottleneck, post-training methods — chain-of-thought prompting, RLHF on factual tasks, retrieval-augmented generation — may address most factual errors without further scaling. The paper shows thinking-mode inference specifically improves recall, corroborating this direction. For teams building factuality-sensitive applications, RAG and post-training alignment target the actual failure mode more precisely than additional pretraining data. WikiProfile is publicly available as a benchmark.