ARASH: Adaptive Retrieval And Shot Selection for Tabular Prediction

Details

Tabular Foundation Models (TFMs) like TabPFN deliver strong predictive performance on structured data without per-task training, but their resource demands scale badly as context grows. In-context learning through few-shot examples is a cheaper alternative, but selecting which rows to include as examples remains unsolved for tabular data. ARASH (Adaptive, query-specific Retrieval And Shot Selection) tackles this by analyzing each query's local neighborhood in the training data to identify the most informative examples. The method wraps around existing TFMs without requiring any retraining or fine-tuning. The paper reports ARASH reduces TabPFN's prompt length by 1,261.5x and memory usage by 2.56x—substantial efficiency gains for production deployments where inference cost is a constraint. Accuracy remains comparable to the full-context baseline, suggesting the local neighborhood captures most of the relevant signal. The approach is particularly relevant for enterprise teams where tabular data workloads are common and inference budgets matter. It provides a plug-in efficiency layer rather than a new model architecture, lowering the barrier for teams already using TabPFN.