Det är lätt hänt att man i arbetet med en regressionsmodell så att säga ”slänger in” alla oberoende variabler man kan hitta. Oftast tillför de någon förklaringskraft, och man kan då förledas att tro att modellen blir mer tillförlitlig. Men det ökar risken för att man i för stor utsträckning passar sin modell till just det urval man har, och inte till det generella samband man vill undersöka. Här är ett utmärkt blogginlägg från bloggen Fivethirtyeight på New York Times, där Nate Silver diskuterar just det problemet.

The problem with an overfit model is that, because it is so fussy about handling past cases, it tends to do a poor job of predicting future ones. Imagine that I was a petty criminal of some kind, and that I deputized you to come up with a way to help me pick combination locks. I also gave you three locks to experiment upon.

What I’d really be looking for would be some set of principles on how one picks locks: perhaps a certain type of paper clip works especially well, or a disproportionate number of combinations contain numbers like ‘7’ and ‘13’. Instead, after studying the issue for a few days, you report back to me that you’ve found the perfect solution. If the lock is blue, use the combination 45-12-26. If it’s red, use 33-9-16. And if it’s black, use 22-10-41. That would certainly be a very reliable way to pick these three particular locks, but it wouldn’t tell me anything about how to pick locks in general. This is essentially the same thing that happens when one produces an overfit statistical model.