Hi, is this IBM customer support? I have the 3 keys you sent me. Do they open something?

IBM, in what is probably not a new experience for them, have developed something kinda science-y: IBM’s “Keys to the Match”, for enjoyment with tennis.

If you go here, you can see post match analysis like this:

keys2

You have to admit, it looks pretty impressive. And yet, questions remain. Quite a few, actually.

If you dig around on the web, you eventually come across IBM’s explanation for KttM:

ibm

Some of the KttM make intriguing reading, too. Sam Stosur, in her third round match against Ana Ivanovic (sorry, against Serbian Glamour Girl Ana Ivanovic), is advised to “average between 5.5 and 6.2 points per game returning”. (Would it be worse to make Ivanovic play longer serve games? Why? Is it a stamina thing? Is Ivanovic’s stamina inferior to Stosur’s? Supplementary curious question: what are “comparable play styles”?)

During matches, IBM publishes the KttM in the Grand Slam Trackers, giving you a view like this:

tracker

As the match progresses, you can keep an eye on how players are performing against the keys. At the end of the match, you can see whether a player reached, exceeded or failed to achieve each of his or her 3 keys.

And then, after that, nothing.

Am I alone in asking a small question, like: what is the point of all this?

And some narrower questions, like:

Does achieving all 3 keys frequently result in victory? Does achieving more keys than your opponent frequently result in victory? What if both players achieve all 3 keys? Does it make a difference if a key is barely missed or barely achieved? What if one key is vastly exceeded and another vastly failed – do they cancel each other out? Has a player ever won a match, even though they have failed all 3 keys and their opponent achieved all 3? Does similar performance against the keys result in a close match?

What evidence does IBM have that the 3 keys are meaningful? What evidence would be required to conclude that the 3 keys are not meaningful?

IBM has tracked 41 million shots (or something). I don’t have quite that much time – so here’s some data from round 2 of the men’s singles at the Australian Open this year, sorted by [Key differential]. There are only 22 matches – not every match is special enough to qualify for the KttM treatment.

keys

It’s hard to get too much out of this except that gathering up those keys seems like a good idea.

The average of [key differential] minus [games differential / 4] is close to zero (with standard deviation of 1 – which seems quite high in this context[citation needed]). A superficial reading of this small sample would be that each additional key a player has above their opponent is worth about four games – well and truly enough to win a grand slam tennis match.

However, none of this proves that the KttM are anything other than a random selection of things that all players want to do all the time. After all, “winning 65% of first serves”, “winning 55% of medium rallies” – aren’t these just proxies for “winning points” and, ultimately, “being good at tennis”? If we randomly selected a bunch of keys associated with winning points, would we do any worse / better?

Don’t get me wrong. IBM has a shedload worth of data to work with, and more than enough brainy people to do something interesting with it. But now they’ve wound up this machine and it’s spitting out all this awesome seeming stuff. Is it rude to whisper to each other behind our hands: “Psst – is this thing working?”

Advertisements

Click here to like, or tell me whether this is genius or gibberish

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s