Stats-abusing journos, the odds are against you

Reading the sports pages and getting annoyed is a familiar feeling for anyone who follows a team or a sport. I can live with reading about my team getting thrashed or being on the wrong end of a bad umpiring decision, but there’s one thing I just can’t stand – being treated like an idiot. Is it too much to ask – for sports writers not to refer to “the law of averages”? Not to supply an average when a median would be a better measure to support (or test) their point? Not to use selective endpoints to skew a tenuous analysis?

Freakonomics, Moneyball and other books (“now Major Hollywood films!” … well, Brad Pitt is in Moneyball) have started to educate the public about how common sense, even supported by statistics, is sometimes just plain wrong. This blog will examine opinion and analysis pieces in the sports media, which have – or try to have – a statistical focus.

Of course, some articles don’t even quite make it over the common sense threshold. Round-one losers, the odds are against you sits at the top of the league table for infuriation. The phrase that set me off was “statistically significant”. (A commenter when The Age originally published this may have done the job I’m about to do by simply writing: “Statistically significant. Bwah-ha-ha!!!” But I’ll persist.)

The author’s hypothesis is that an AFL club’s first game of the season holds special predictive power – a loss dooms the club to a premiership-free year. How does he know this to be so? Because the opposite is false – it didn’t happen 35 out of the last 50 times. So if the opposite of a proposition is false, the proposition must be true… right?

[Gratuitous note in an ironic font: why is the author so anti-loss? Premiers’ round one results included 14 losses and only 1 tie. Only 1 tie in 50! So, losing might not help your premiership chances, but clearly a tie is the death knell.]

The problem (ok, one problem) is the author’s poor definition of the null hypothesis. A typical null hypothesis is that you’d expect a tossed coin to come down heads half the time, or a rolled dice on six (or five, or two) 1/6 of the time. With a juicy set of data you might set out to show that a dice is loaded towards landing on a five. So, your null hypothesis is that the dice is just a normal, regular, non-cheating dice – the sort that won’t get you beaten up if you’re gambling money on it, the sort that lands on five 1/6 of the time. Your alternative hypothesis is that it’s weighted like a lawn bowl, and five comes up more than 1/6 of the time. A bit like a prosecutor with the dice on trial, though, you have to have decent proof – usually 95% confidence or more in the data.

If your data don’t give you that (pause here for smug enjoyment of using data as a plural noun) you’re stuck with the null hypothesis. The success of a null hypothesis basically means, “Wow. This experiment has been a real waste of time.”

So, what would be the null hypothesis in this situation? What do we expect to happen to premiership clubs in their season openers? Well, the author gets pretty excited about a win rate of 71% (if a tie = half a win), so he must be comparing it to something much lower. He never comes right out and says it, but (fight urge to cringe) “statistically significant” makes me think he reckons we should be comparing it to 50% – the flip of a coin.

Hmm. You see, what I’d expect is that they’d win games with the same relentless efficiency that they’re about to show for the rest of the season. Using the same period as the article, and counting ties as half a win, premiership teams won 803.5 out of 1052 home and away games – 76.3% (or 76.6% excluding all round one games). This means that over the balance of the season, it’s likely that more than half of the rounds have a better win rate than round one (and surely all would be sufficiently higher than 50% to earn the “statistically significant” tag). So why not write, “Round Two Losers, the Odds Are Against You”, “Round Three Losers, the Odds Are Against You”, “Round Four ….”. Or perhaps it would be simpler just to write, “Losers of any AFL match, the Odds Are Against You”. Given that premiers win over three-quarters of their matches, any time you look at the weekend’s crop of losers, they’re more likely to be teams destined to fill end of season slots two and downwards.

Meanwhile, it looks as though round one performance, rather than being critical to the year’s showing, is actually worse. What’s going on?

Nothing particularly interesting. With enough data (and this is plenty), a mathematical formula with the somewhat understated name z-test can tell you with a specified level of confidence whether two experiments (premiership teams playing their season opener / premiership teams playing other games in the home and away season) really do have different probabilities of success. With 95% confidence the z-test tells us that the true value for first-round success is somewhere between 58.4% and 83.5%, and the rate for round 2 onwards between 73.9% and 79.2%. The first range totally encompasses the second, so there’s no way you can confidently differentiate the two. (And you’ll have noticed how wide the first range is – the less data, the less precision about the true value – even with a sample size of 50. Keep that in mind next time a commentator is salivating over “the Cats have beaten the Bombers 4 out of the last 6 meetings …”. Yeah. Come back when you have some real data.)

Or, using a slightly different z test, you can say with 99% confidence that the difference between the two success rates could be as high 24.1 percentage points, or as low as nothing at all. Despite it all, I’m sure that some people who read the data would insist that premiership teams’ performances really do improve post-round one. What could be the reasons for this? Growing confidence? Team spirit? Player development? Destiny?

My preferred reason would be sampling bias. The underlying assumption for everything about these calculations is that all games pose an equal challenge: ie, beating the 1996 Fitzroy Lions is just as hard as beating the 2001 Brisbane Lions. Clearly this is rubbish – just ask Martin Pike (possibly in your most polite voice). Teams which are about to win a premiership are usually pretty good already; the AFL likes round one to include good teams playing against other good teams. That means a higher likelihood a tough opponents and a lower likelihood of wins. This is just one reason why round one is different – there may be many others.

But we’ve gotten off track. Let’s go back to the, ahem, “statistically significant” claim that started this all off. If normality would be premiers only winning half their season openers, would this data disprove it?

Ok. Now we’ve got ourselves a headline. The z-test gives us a high – off-the-charts! – confidence level that premiership teams win better than 50% in their season openers. 50% is more than 3 standard deviations away from the mean (or average) of this data – a confidence level of over 99.8%. For comparison, Don Bradman’s batting stats are about 5 standard deviations away from overall test averages. So, we’re basically good to go with this conclusion.

And… the same holds true for all games by premiership winners!

It’s a heady morning for maths. We’ve just proved that premiership teams – with barely the shadow of a doubt – win more than half their games. Would you like to call the AFL, or should I?

Click here to like, or tell me whether this is genius or gibberish

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s