Thinking Fast, Trading Late: System One Decision Models vs the Market

| Source: arXiv AI

Tags: OLIVIA, LLM Trading, S&P 500, Financial NLP, Pre-Registration

A pre-registered test finds LLM news signals for S&P 500 stocks mostly vanish out of sample, with a fine-tuned model's information coefficient falling to 0.020 and no strategy surviving costs.

Details

Backtests often suggest LLM readings of company news predict returns. The OLIVIA protocol tests that claim strictly: point-in-time inputs with company names masked, selection on validation data only, an analysis plan frozen by hash and a test period unblinded once. A fine-tuned open-weights model scored daily information coefficients of 0.055 and 0.065 on validation quarters but 0.020 (t = 1.7) on the blinded January to July 2026 test. Commercial models line up with the overnight move to the open (0.074 to 0.077) but not with moves after it, so the news is priced before a trader can act. No pre-registered strategy survived costs and a deflated Sharpe ratio, and commercial models identified masked companies in 62% to 74% of cases against a 28% baseline.