Who Questions What Works: When Should We Retest Our Assumptions?

Details

The piece distinguishes two failure modes that share a common root. Kodak failed to adapt at all. Moody's, more instructively, kept adapting models and methodologies as markets evolved — yet remained bounded by assumptions already embedded in its rating system. The author argues this is the more dangerous failure mode: continuous adaptation that never questions the original problem framing. In data science, the dynamic is familiar. A hypothesis survives testing and becomes a model. A model performs well and becomes a system. A system repeated long enough becomes doctrine. At each step, the evidentiary bar for challenging it quietly rises. The article asks a pointed question: how often do we retest the assumptions behind something that still seems to work? The piece is light on concrete methodology — no audit frameworks, no statistical tests, no tooling recommendations. It reads as a framing piece rather than a practitioner guide. But the underlying concern is legitimate, particularly for LLMs deployed in enterprise settings where data distributions, user behavior, and business context shift gradually without triggering obvious performance alerts. The Don Quixote epigraph sets the theme well: the risk is not just mistaking windmills for giants, but building an entire epistemology around the assumption that the landscape is full of giants.