Update: read more on this here.
Yikes. Pretty hard to put a positive statistical spin on any of that.
The prevailing sentiment is that a soccer goalkeeper is rarely noticed, or at least rarely remembered, unless they’ve just something unspeakably awful. That might be reason enough not to get out of bed in the morning, but it might be just as frustrating to the GK that the average fan – and maybe even the above average fan, or worse still, the coach, or even the GK – doesn’t understand whether a GK has had a good game or not.
Part of the problem is the lack of data to evaluate performance. Athletes who play cricket, baseball, Australian or American football can point to a raft of numbers. Soccer GKs have only shots, saves and goals, and not very many of any of these in one game. Add to that that not all shots, saves and goals are equal – or easily described – and all the fan is left with to make a judgement is images like those above, or conversely, like those below, which will be blunted by all sorts of cognitive biases – to name just a few: attentional bias (focusing too much on recent data, or not enough or data to which you have no emotional attachment; in soccer terms that might mean ignoring GK actions that had little bearing on a match’s outcome), confirmation bias (remembering more strongly that which you already agree with), and clustering illusion (irrational belief in “hot” or “cold” streaks). And that’s just going alphabetically from A to C.
But the Guardian’s Sean Ingle writes in some detail about Roy Hogson‘s Joe Hart / Fraser Forster national team GK selection dilemma in England, and existing systems to monitor and evaluate goalkeeper performance. There’s certainly a lot more data about than I was aware of, and a lot more than gets regularly reported in any of the major soccer media sites, like Soccernet or Yahoo! Sports.
Examples of data collected are not just shots and saves (from inside and outside the penalty area), but “mistakes” and saves from “big opportunities”. Sounds fascinating. But the tenor of the article and some of the reader comments that follow is, “Crikey isn’t that a lot of data… If only that would help me choose between Hart and Forster.”
The data collection seems to be very much on the right track. But then there is a statement like this:
“Bloomberg’s Sport has also used another Opta metric – keepers’ mistakes – across the major European leagues since 2009-10. On this measure, Hart performs moderately well. He has made 15 mistakes leading to a goal or shot during this period – the same as Petr Cech and better than Victor Valdés (16), Manuel Neuer and Iker Casillas (17), but behind Hugo Lloris and Gianluigi Buffon (nine) – and makes a mistake leading to a shot or goal every 1,026 minutes, again better than average.”
Really? Who’s likely to have more mistakes – the GK for a club at the bottom of a league, who sees the ball, say, 30 times a game, or at the top of the league, who sees it 15 times? I have no idea what the actual proportion is. But surely this would be a useful thing to count, if you’re going to bother counting everything else mentioned in the article.
The confusion seems to stem from not understanding what is being analysed and why. Leaving aside the idea of the GK as “first line of attack”, the defensive function of the GK in soccer is to respond to situations where:
1) An opportunity to shoot at goal may soon occur
2) A shot at goal has occurred
Several situations like this will arise in any game; the GK’s job is to minimise the number of these that result in goals. (Rewrote several times and tried to make it sound drier. Couldn’t.)
Using save percentages and mistakes, Ingle describes the outcomes of a number of these events; all that’s missing is to synthesise it into the crucial output – how many goals will result?
In my last post I explained what E(X) = n.p means. How exciting that we get another chance to use it so soon!
Let’s try an analogy. Playing casino games is a good way to (eventually) lose money. Playing as GK is a good way to (eventually) concede goals.
The casino offers cards, dice and the roulette wheel. How much money can you expect to make (or lose) by betting $10 on a few games of each? Gamblers are masters of cognitive bias – but we can use maths instead.
The calculations behind this are pleasingly straightforward. To find the expected result for playing one game (which is most easily understood as simply meaning “long term average per game”), you just multiply the likelihood, p(x), of each available outcome by its profit / loss, and add up the results.
Let’s start with roulette. The roulette wheel has slots zero to 36. It pays 35 times the bet amount for picking the right number, or double your money for red or black. Even losing this way is fun. Just ask Bart Simpson, shown here in the original German:
But, fun or not – and whichever way you bet $10 – you can look forward to slowly losing your money, 27 cents at a time.
The card game is Texas Hold ‘Em. There are five other players at the table, some of whom you know to be inexperienced. The $10 bets slowly build up. You know that you’ll have a slight edge when there’s a small pot (you contribute $10 to a pot of $60), and a better edge on the large pots (you put in $100 to a pot of $600). Meanwhile, the house takes 5% of every pot.
Not a bad game to be playing – you’ll slowly liberate the dupes from their wages, even with the house taking its cut (in return for making sure they don’t beat you up on the way to the car park).
The dice game is craps. The rules of craps are complicated, so I’ll simplify it to the basic bet, cobbled together from wikipedia. Two dice are continuously rolled:
If the first roll is 7 or 11, the bet wins.
If the first roll is 2, 3 or 12, the bet loses (known as “crapping out”).
If the roll is any other value, it establishes a point.
If, with a point established, that point is rolled again before a 7, the bet wins.
If, with a point established, a 7 is rolled before the point is rolled again (“seven out”), the bet loses.
(I’m not sure I’ve quite got these rules right. Let’s just say these are the rules for now. Craps enthusiasts, feel free to write in.)
The tables below show what will happen once a “point” is established at various values. The game ends only once a 7 or the original value is rolled – you’ll lose more often than you win.
These expected values are then fed into the table below, representing all the possible outcomes of the initial throw.
So, craps proves to be another way of continuously losing your money in under-couch-cushion denominations.
What happens when you play a game of each? You can multiply these by the number of games to get the expected total profit or loss from a gaming session. If you played 10 games of each, the expected outcome is 10 x -0.14 + 10 x 9.81 + 10 x -0.27 = $94. Nice work!
How does this help us calculate GK effectiveness? GK match involvements are each like a small game. The list of games to play might look something like this: crosses, through balls, close and distant shots, and clear-cut opportunities. The result of each game might be a goal, another close or distant shot, or another cross – everything else is “not a goal”. Like the game of craps, the sequence continues until it ends in a goal, or not.
Crucially, you also need to know how many crosses, shots etc happen in an average game. (Actually, you don’t need to know how many, just in what proportion they happen to each other.) This will dictate the weighting given to each of the different games, and GK skills in making the games end in “not goal”.
The cross / shot / etc proportions may differ from league to league. I’d be surprised, for example, if lower-league matches didn’t feature more through balls (although “through balls” might be a lofty description for hoofing the ball over the top of the defence and chasing after it). The English Premier League used to have a reputation for being cross-happy, but I’ve never seen any evidence.
So, let’s assume that the per game occurrences are 15 through balls, 8 crosses, 3 shots from distance, 1 shot from close range and .5 of a clear chance (ie, once every 2 games).
If Joe Hart, for instance, concedes a goal from 1% of through balls, 7% of crosses, 12% of distant shots, 27% of close shots and 75% of clear chances, he’ll concede 1.72 goals per game. (Sounds a little high. Sorry, Joe.)
You could make this a little more sophisticated by creating a number of outcomes for each GK involvement – for example, as well as just a goal or not-goal, a distant shot could also result in a cross (eg – GK parries the ball sideways, but to an attacker), or a close-range shot (parry close to goal) or a clear opportunity (GK fumble or other stuff-up). That would make the table appear a little more like the craps table, with some outcomes feeding into others. A GK like Gianluigi Buffon, who I’ve always thought parried the ball a lot rather than catching it cleanly, might have a good save percentage from distant shots, but generate more crosses – which in turn might generate more shots and / or goals. (These Buffon thoughts probably reflect my own cognitive bias.) If shots result in crosses and crosses in shots, it might make an interesting recursive exercise.
The weakness of the whole approach is the subjectivity of identifying each of the GK involvements. Soccer is a fluid game, and it’s not always clear where a piece of action begins or ends. There is also room for some slightly finer distinction between easy, average and difficult GK tasks in each of the involvements.
The weakness of not using some sort of approach like this is that cognitive bias will throw you off the scent of the best GK. Decoy effect (another GK gets a lot of press, out of proportion to his performances), duration neglect (inability to accurately take into account how frequent good / bad performances actually are)… not even past D yet!
Roy Hodgson’s Wikipedia entry includes no mention of an interest in cognitive biases. Surely this is an oversight? I’ll be expecting a call soon.Follow @newstatsman