There has been lots of talk this year that Manchester United aren't playing the most beautiful or inspiring football, but that they owe their table position instead to playing well week in and week out. Really well, mind you, at least according to some of the metrics I've looked at, but they're not necessarily no.1 in every category.
Measuring team performance is tricky business. So what would be a good way to get a handle on whether teams like Man United reliably perform at a high level? It strikes me that you want to see a combination of two things: high levels of team performance, but with low variation around that high level. For example, you want your team to be able to generate, say, 12-14 shots in a match, but you also want them to do that every time, rain or shine, home or away, no matter what. It doesn't take much to imagine that teams that can do these two things are likely to contend for the title. In contrast, teams that have too many ups and downs - a glorious 5-1 victory followed by an agonizing 0-3 - will not be able to win points every time as the league leaders typically do.
Statistically speaking, what we want to do is put a number on the level of performance as well as the variation around that level of performance over some period of time. For starters, imagine a distribution of performance over a couple of months or a season. Offensive production in the form of shots can serve as an example. Here's the distribution of shots in the EPL in the 2009 season by team and match. It shows you how often teams performed in the range indicated.
Average team performance in the 2009/10 season was 12.2 shots per match. But what this number doesn't tell you (but the graph above does) is that there also was considerable variation around that mean. While performances in the center of the distribution were most common (in the 7-13 shot range), there also were a good number of times when teams hardly could hardly shoot straight or when they shot their opponents' lights out.
To see if teams' performance levels were dependably in a high or low range or all over the place, we can calculate a statistic called the standard deviation.
Here's how Wikipedia explains the concept: "Standard deviation is a widely used measurement of variability or diversity used in statistics and probability theory. It shows how much variation or "dispersion" there is from the "average" (mean, or expected/budgeted value). A low standard deviation indicates that the data points tend to be very close to the mean, whereas high standard deviation indicates that the data are spread out over a large range of values." (A slightly more mathematical explanation is here.)
If you're not interested in the statistical gobbledygook, here's the intuition. By calculating the standard deviation around the mean, we can see if teams are able to exceed the mean and whether they do so consistently (or have really high performance followed by really low ones). The worst kind of performers will be teams that have a low mean (teams that, on average, perform poorly) and do so consistently (i.e., they're bad every match). In-between, you'll have good teams with a fair number of bad days or bad teams that occasionally kill their opponents.
To illustrate the concept, here are two graphs. The first one represents a team's average number of shots and the standard deviation. Take a look.
The graph reveals a couple of things. First, it shows one piece of information we've already known from analyses of match averages: Man United, Chelsea, and Liverpool produced the highest average numbers of shots per match around, while Stoke did about half as well. We also see that Hull, Fulham Sunderland, and Wolves all performed about equally well by producing about 10 shots per match. However, the standard deviation also shows that Wolves did so with much less consistency than the other three teams. It also reveals that Stoke's performance was dependably lousy: they didn't shoot much and managed to do so match after match.
Here's a potential shortcoming with the graph: what it can tell us may have to do with how we are measuring (counting) performance. You can see that the graph reveals a positive correlation between mean shots per match and their variability: teams that shoot more also have more variation around their performance than teams that shoot less frequently. Chelsea's higher performance, which ranged from 7 to 29 shots per match, also produced more variation, while Stoke's shots ranged from 2 to 15, and thus produced a lower standard deviation. This is perhaps not surprising; it's hard to sustain truly amazing performances (with 25 plus shots per match), so there will be more variability around the performance of a Chelsea because it can't sustain the very highest levels week in and week out.
But to make sure that the results aren't driven by the metric we're using, it helps to find a way to standardize performance (and hence make things more comparable). Think about it this way. A simple count of the number of shots per match can theoretically range from zero to a very high number (when Chelsea play my son's U10 team one week and then play Man United the next), and these higher and lower numbers are likely to produce more variability. So instead, we can also measure things on a scale that always ranges between, say, 0 and 1. A good example is accuracy - the ratio of accurate to the total number of shots. This gives a slightly different picture; take a look.
 |
| A picture of consistency |
Statistically speaking, what we want to do is put a number on the level of performance as well as the variation around that level of performance over some period of time. For starters, imagine a distribution of performance over a couple of months or a season. Offensive production in the form of shots can serve as an example. Here's the distribution of shots in the EPL in the 2009 season by team and match. It shows you how often teams performed in the range indicated.
Average team performance in the 2009/10 season was 12.2 shots per match. But what this number doesn't tell you (but the graph above does) is that there also was considerable variation around that mean. While performances in the center of the distribution were most common (in the 7-13 shot range), there also were a good number of times when teams hardly could hardly shoot straight or when they shot their opponents' lights out.
To see if teams' performance levels were dependably in a high or low range or all over the place, we can calculate a statistic called the standard deviation.
Here's how Wikipedia explains the concept: "Standard deviation is a widely used measurement of variability or diversity used in statistics and probability theory. It shows how much variation or "dispersion" there is from the "average" (mean, or expected/budgeted value). A low standard deviation indicates that the data points tend to be very close to the mean, whereas high standard deviation indicates that the data are spread out over a large range of values." (A slightly more mathematical explanation is here.)
If you're not interested in the statistical gobbledygook, here's the intuition. By calculating the standard deviation around the mean, we can see if teams are able to exceed the mean and whether they do so consistently (or have really high performance followed by really low ones). The worst kind of performers will be teams that have a low mean (teams that, on average, perform poorly) and do so consistently (i.e., they're bad every match). In-between, you'll have good teams with a fair number of bad days or bad teams that occasionally kill their opponents.
To illustrate the concept, here are two graphs. The first one represents a team's average number of shots and the standard deviation. Take a look.
The graph reveals a couple of things. First, it shows one piece of information we've already known from analyses of match averages: Man United, Chelsea, and Liverpool produced the highest average numbers of shots per match around, while Stoke did about half as well. We also see that Hull, Fulham Sunderland, and Wolves all performed about equally well by producing about 10 shots per match. However, the standard deviation also shows that Wolves did so with much less consistency than the other three teams. It also reveals that Stoke's performance was dependably lousy: they didn't shoot much and managed to do so match after match.
Here's a potential shortcoming with the graph: what it can tell us may have to do with how we are measuring (counting) performance. You can see that the graph reveals a positive correlation between mean shots per match and their variability: teams that shoot more also have more variation around their performance than teams that shoot less frequently. Chelsea's higher performance, which ranged from 7 to 29 shots per match, also produced more variation, while Stoke's shots ranged from 2 to 15, and thus produced a lower standard deviation. This is perhaps not surprising; it's hard to sustain truly amazing performances (with 25 plus shots per match), so there will be more variability around the performance of a Chelsea because it can't sustain the very highest levels week in and week out.
But to make sure that the results aren't driven by the metric we're using, it helps to find a way to standardize performance (and hence make things more comparable). Think about it this way. A simple count of the number of shots per match can theoretically range from zero to a very high number (when Chelsea play my son's U10 team one week and then play Man United the next), and these higher and lower numbers are likely to produce more variability. So instead, we can also measure things on a scale that always ranges between, say, 0 and 1. A good example is accuracy - the ratio of accurate to the total number of shots. This gives a slightly different picture; take a look.
Remember that, on this kind of metric, we want to see teams do well on average (have a high mean) and do so consistently (have a low standard deviation): so you want to end up in the upper left hand corner. And we don't want to see consistently low performance, so you don't want to end up in the lower left hand corner where the average and the standard deviation are low.
As the graph shows, the very best teams last year did indeed inhabit the upper left hand corner: Man United, Arsenal, and Chelsea all had high levels of accuracy and some of the lowest levels of variation around their high accuracy levels. Here are some other illustrative examples. Wigan consistently could not shoot straight, and Man City, with all its investments, cut a very middling figure all around. Wolves were a true outlier: they had high levels of accuracy (when they finally got around to taking a shot, mind you), but their performance was truly erratic.
So what do things look like this season? here's a teaser (I'll tell you more in a future post), using the goal to shot (or Reep) ratio (it tells you how many goals teams manage to score per shots taken or how many shots it takes to score a goal). Here are means and standard deviations on this metric as of the end of February.
Liverpool is the most consistent performer this season, albeit at a significantly lower level than teams at the very top, and similar to Tottenham and Bolton. Wigan and West Ham are among the consistently low performing teams (and Aston Villa and Fulham aren't all that far away from them). Finally, Arsenal has a lower average, but also more reliable performances than Man United who have the best efficiency ratio in the league, but with slightly more volatility. To me, it looks like Arsenal and Man United are the two teams populating the "high performance, low variability" area of the graph. And with about a third of the season to go, Birmingham stands out as the most erratic performer of the season (on this score), and by a mile.
The point here is not to equate graph position with eventual league position, though that may be something worth exploring (check out the latest posting on Onfooty.com on offensive performance and points). Instead, it is to make a basic conceptual point. (And, by the way, these numbers don't take into account how many shots teams generate per match, which is why Blackburn look like Man United on the average Reep ratio per match).
So while it's not exactly in the eye of the beholder, team performance can be measured in different ways, and different people may want to gauge different aspects of how a team is doing. To find out if a team is doing well, averages are often good enough. And maybe averages even win titles (if they're high enough, that is). But I'd also bet that high performance each and every time is even better!
