ATTRICITE: Training an Open 4B Model for Citation Recovery toward Faithful Attribution
| Source: arXiv AI
Tags: citation-attribution, Qwen3, GRPO, RAG, open-source-model, scientific-AI
ATTRICITE, an open 4B-parameter model fine-tuned via GRPO on Qwen3-4B, improves scientific citation recovery accuracy from 49.4% to 59.8% on held-out 2025 CS papers — outperforming gpt-oss-20b and landing within 3.9 points of GPT-5.4-mini at a fraction of the parameter count.
Details
Citation attribution — ensuring AI-generated scientific text cites the correct sources — is increasingly important for RAG systems in research contexts. ATTRICITE approaches this as a citation recovery task: given a citation-bearing passage, recover the paper the original author intended to cite. The model is built by GRPO (Group Relative Policy Optimization) fine-tuning of Qwen3-4B within the CiteGuard retrieval environment, trained on CITEALIGN, a 7,607-instance dataset drawn from recent CS literature. On a 299-instance temporally held-out test set from 2025 publications, GRPO fine-tuning improves accuracy from 49.4% ±1.5% to 59.8% ±0.2% — a 10.4 percentage point gain. ATTRICITE outperforms gpt-oss-20b despite being roughly 5x smaller, and comes within 3.9 points of GPT-5.4-mini. Gemma 4 31B IT achieves the best overall performance at 72.0%, setting the upper bound. The model and data collection pipeline are released publicly. This is labeled 'Work in Progress' and the dataset is restricted to computer science — generalization to other scientific domains would require new data. Real-world citation complexity in continuously evolving literature likely exceeds controlled benchmark conditions.