GRPO Beyond English: A Large-Scale Study of GRPO in Non-English and Multilingual Settings
| Source: Apple ML Research
Tags: GRPO, RLVR, multilingual LLMs, Apple ML Research, reinforcement learning, cross-lingual transfer
Apple ML Research finds that GRPO reasoning training in a model's native language closes most of the gap to English-only training, with strong cross-lingual transfer across languages — but warns that some configurations cause silent capability regressions that only broad evaluation can detect.
Details
Group Relative Policy Optimization (GRPO) has become a leading technique for improving LLM reasoning via reinforcement learning with verifiable rewards, but nearly all published work trains exclusively in English. Apple ML Research fills this gap with a large-scale empirical study spanning multiple base models, training languages, and reward configurations.\n\nKey findings: training to reason in a model's native language typically leaves only a small gap to the English-trained baseline, meaning non-English users do not need to accept significantly worse reasoning performance. Strong cross-lingual transfer is also observed — training GRPO in one language often improves performance in several others without explicit multilingual training data.\n\nHowever, the picture is model- and language-dependent. In some configurations, GRPO training in a particular language causes severe regressions in out-of-domain capabilities for other languages — a failure mode that would be invisible without comprehensive multilingual evaluation coverage. The authors explicitly advocate for broad evaluation whenever non-English GRPO is applied.\n\nFor teams building non-English AI products or fine-tuning reasoning models for specific language markets, this provides the first systematic evidence that GRPO can be applied in native languages with competitive results — along with practical guidance on what can go wrong.