What the Reranker Sees: Multi-Aspect Page Annotation for Long-Document Multimodal Question Answering

Details

Retrieve-then-read pipelines for long-document QA have a known bottleneck: high recall at the retrieval stage does not translate to good evidence selection. On MMLongBench-Doc, BGE-M3 achieves Recall@20=0.86 but only F1@5=0.254 — the right pages are in the candidate set, but the reranker selects wrong ones. Trident addresses this with two components. Trident-R converts each retrieved page into a structured semantic record — a visual caption, section path, entity tags, multi-axis concept hits, and a text snippet — then performs a single adaptive-K reranking call. The structured annotation is the key: an LLM reranker seeing only raw snippets barely changes first-hit ranking, while the same reranker with structured annotations substantially improves retrieval F1 across five heterogeneous retrieval pools. Trident-S, the generation-side module, prompts the vision-language model under topical, entity, and structural lenses before synthesis — adding up to 6.6 points in generation accuracy on open-ended synthesis questions. Both components are retriever-agnostic and compatible with any upstream system. The best Trident configuration is the strongest downstream QA pipeline in the evaluation. Cross-judge agreement is strong (kappa=0.913 between two LLM judges), adding confidence in the robustness of results. This is a practically deployable improvement for enterprise document intelligence pipelines.