Coverage-Maximizing Multinomial Subset Routing under Operational Constraints

Details

Multinomial Subset Routing (MSR) addresses a gap in existing online learning frameworks: how to route queries across K expert models when the reward comes from the best-performing model in a sampled subset, not from the sum of all selected models (as in combinatorial bandits). This non-additive reward structure arises naturally when deploying ensembles of specialized AI models.\n\nThe key innovation is maintaining a multinomial policy rather than a deterministic subset. At each round, M experts are sampled i.i.d., and the reward depends only on the winner. The authors propose OMD-Approachability — combining online mirror descent with Blackwell's Approachability — and prove it achieves O(1/√T) regret in both reward and constraint violation under bandit feedback.\n\nThe framework also handles long-term, two-sided operational constraints, relevant to deployments where cost, latency, or fairness requirements must be maintained over time. The paper validates the approach on a real-world crowdsourcing dataset.\n\nFor ML practitioners building routing layers over multiple specialized models, MSR provides a theoretically grounded approach that standard bandit methods miss. The practical applications — model routing, crowdsourcing, recommendation — are commercially adjacent.