TactiNavi: Serving and Revising an LLM Game Coach with Explicit Strategy Conditions
| Source: arXiv AI
Tags: TactiNavi, LLM serving, game AI, decision traces, release validation, LLM evaluation
TactiNavi, the serving system behind the LLM coach in a commercial open-world game, makes strategy prerequisites explicit and lifts correctness on logged coaching turns from 57.9% to 82.0%, while showing most apparent fixes were just generation noise.
Details
An in-game LLM coach has to give advice that depends on player state and game version, so an answer that is right for one player can be wrong for the next. TactiNavi compiles versioned game evidence and expert guidance into strategy packages with explicit prerequisites. At serving time those prerequisites are checked as three-valued conditions over structured player state, and decision traces are recorded. A control plane uses condition-rule dependencies to validate revisions on held-out traffic before release. On 1,122 logged coaching turns from three deployment periods, condition-bearing packages raised correctness from 57.9% to 82.0% and cut strategy misapplication under changed state from 18.1% to 10.4%. Trace-based claim verification hit 86.4% precision at a 2.07 s median reply latency. The most useful lesson for practitioners: rerunning the unchanged system recovered 38.4% of failing turns, and only one of 15 shipped revision units passed release checks retrospectively, so release pipelines need a null-model baseline.