All posts
Article1 min read

Regression, Correlation, and the Tyranny of the Past

Regression forces a pattern onto data without proving cause. Correlation dressed as insight. Most predictive analytics just replay the past forward. We call it prediction; it is mostly extrapolation.

By Shailin Dhar

Regression analysis is a machine for drawing a confident line through a cloud of dots. It always finds a line. That is the problem. The math does not care whether the relationship is real, causal, or stable. It fits the pattern you asked it to fit and hands you back a slope with a reassuring number attached.

What you get is correlation dressed as insight. The line describes how two things moved together in the data you already had. It says nothing about why they moved, and nothing that guarantees they will keep moving that way. Yet the output looks like knowledge, so we treat it like knowledge.

Then we compound the error. Most so-called predictive analytics is just a regression of the past, projected one step further. The model learns yesterday and assumes tomorrow will rhyme. We call it prediction. It is mostly extrapolation, a confident continuation of the training data, and nothing more.

A model that has only ever seen the past can only ever bet that the past continues. That is a wager, not a forecast.

This matters precisely when it matters most: when the future is not a continuation of the training data. Regime changes, new fraud tactics, a novel platform, a shift in incentives: these are exactly the moments the extrapolator is blind to, because they were absent from the history it fit. The tyranny of the past is that it forecasts confidently right up until the world stops obeying it.

MeasurementMetricsAI

Keep Reading

AI-assisted recommendations of related work elsewhere on the internet.