BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models

Details

Most multimodal benchmarks test general visual comprehension or lean heavily on English and Chinese. BEAR-Bench (Bilingual Enterprise and Academic Reasoning) targets a different need: reasoning over text-dense professional documents — financial reports, scientific papers, contracts — where models must synthesize complex structured content rather than extract visible facts. The benchmark covers 1,000 human-annotated questions in English and Russian, evaluated on 16 proprietary and open-weight models including Gemini 3.1 Pro and Qwen3.5-397B. The paper reports clear headroom — even the strongest systems fall short — though specific accuracy figures are not quoted in the abstract. Beyond model evaluation, the benchmark is also used to compare hallucination detection methods: how often models fail, and how reliably those failures can be identified. This is a useful secondary contribution for teams building document-heavy enterprise pipelines. The Russian-language component is a deliberate choice to address a gap in multilingual evaluation. Enterprise deployments in Eastern European markets often need model performance assurances on Cyrillic-script professional content that no existing benchmark provides.