TAPES: Task-Aligned Joint Predictive Pretraining for Transaction Event Sequences

| Source: arXiv AI

Tags: TAPES, self-supervised pretraining, transaction sequences, fraud risk, fintech

TAPES picks pretraining targets for transaction-event sequences by their measured downstream value, then pretrains jointly with a shared discrete latent, beating baselines on a real transaction-risk task and public finance benchmarks.

Details

Transaction sequences in commerce and fintech feed many prediction tasks, and the same records can yield targets about values, changes, or relations to past events. TAPES builds candidate target sets from field-operator combinations, runs short matched-budget pretraining trials, and picks the set whose representations help downstream tasks most. It then reinitializes the encoder and pretrains on the chosen targets with a joint likelihood tied together by a shared discrete latent variable before supervised fine-tuning. The authors report gains over representative baselines on a real transaction-risk setting and several public financial benchmarks.