Sunday, April 17, 2011

See No Evil? Fouls, Cards, and Referees in Major League Soccer

In the wake of recent discussions over the quality of refereeing in MLS and David Beckham's ability to earn yellow cards this season, I thought I'd take a quick look at patterns in MLS refereeing with data from last season. As I mentioned in my comparison of fouls and yellow cards between MLS and the big European leagues on the New York Times Goal blog, so far this season, teams in the Premier League have been called for an average of 11.3 fouls per match, while teams in the other three leagues have been called for a third more (ranging from 15 in La Liga to 16.3 in the Bundesliga). But what is most interesting about the foul statistics is that MLS has by far the lowest foul totals of any of the five leagues I compared, at 9.7 per team/match. 

In terms of awarding yellow cards, MLS refs are the least busy overall, too. MLS teams see an average of 1.51 yellows per match; compare that to the average of 2.64 in Spain’s La Liga, for instance. The numbers of yellows in the other leagues are 1.60 for the EPL, 1.67 in the Bundesliga and 1.96 in Serie A. Clearly, we see relatively few foul calls and the fewest yellow cards in the MLS.

These data are useful for putting refereeing in MLS in context. But they cannot tell us whether referee performance is similar across referees, or whether we see distinct patterns in how some referees call the game. Basically, they tell us that play in MLS is either cleaner or its referees pay less attention. I'm sure David Beckham would bet it is the latter; in fact, he apparently thinks his six year old would do a better job.

Below I try to shed a little more light on refereeing patterns in MLS with the help of data from For starters, let's take a look at the league's assignments of referees last year. Clearly, there's a huge range in terms of how many matches referees were asked to call, with a handful of referees calling about 20 matches, and some only getting to call one or two matches. Take a look at the list below.

Clearly, the league has more faith in some referees' ability to do a good job, with Ricardo Salazar topping the list. But the uneven assignment has implications for analyzing referee performance. If we want to take a systematic (statistical) look to evaluate referee performance, we need a big enough sample of matches so that we can draw more reliable conclusions from the data. So for the analysis that follows, I restrict myself only to those referees who were in charge of at least 8 matches. This leaves us with 16 referees we can compare with some degree of confidence. 

How many fouls did MLS referees call? Below is the average number of fouls they called on each team per match last year.

The numbers show a fairly narrow range across referees, from slightly over 8 fouls/team (or 16 per match) for Jair Marrufo to 12 per team (or 24 per match) for Terry Vaughn. Mark Geiger, Kevin Stott, and Jorge Gonzalez are the most average referees, so to speak, at around 9.7 fouls called on each team. For comparison purposes, in an earlier analysis of refereeing in the Premier League I found a much greater range (between 16 and 30). Bottom line: referees in MLS seem to differ little from one another in terms of how many fouls they call.

What about patterns in home-away refereeing? Believers in the home field advantage think that home teams are systematically advantaged. I have my doubts, and my analysis of refereeing in the Premiership did not confirm this. But what about MLS? Do we see some referees systematically calling more fouls on away than on home teams? The following graph shows average numbers of fouls called by each referee on home and away teams.

Turns out, here, too, MLS referees are overwhelmingly similar to one another. More importantly, we do not see any systematic advantage for home teams over away teams in the number of fouls called per match. Yes, there are some referees who tend to call more fouls on the away team. These include Paul Ward, Mark Geiger, Michael Kennedy, and to a lesser extent Hilario Grajeda and Abbey Okulaja (the target of David Beckham's recent wrath when he gave a penalty to DC United in the match against the Galaxy). But by and large, this looks pretty evenhanded to me.

But perhaps the average numbers of fouls called are the wrong metric since the question is whether home teams are advantaged within the context of a single match. So here's the difference in fouls called on the home and away teams in each match (home fouls minus away fouls), by referee (on average). A negative number indicates more fouls called against the away team.

Turns out, it makes no difference how we slice these days. When there is an advantage, it tends to accrue to the home team, but the actual numbers are quite small. We are talking on average about a difference of half a foul per match. Hardly the kind of thing worth worrying about (though we again see the individual exceptions I mentioned above).

So far so good; referees in MLS call relatively few fouls and they do not systematically call significantly more fouls against the away team. But maybe this is not where bias or referee quality comes into play. Maybe referees are by and large similar in which actions they identify as fouls; refs also have relatively less discretion when it comes to fouls. As Muskowitz and Wertheim, the authors of Scorecasting, point out, referee discretion comes into play when they have more power to make calls one way or the other. In the context of fouls, this is most easily the case in situations where yellow cards are awarded. With some exceptions, which fouls count as flagrant or more serious is up to the referee.

So does this mean that MLS referees gave more more yellow cards to away teams last year? Here are the data.

As it turns out, this is where we see a more obvious difference in refereeing decisions for and against home and away teams. Clearly, away teams see more yellow, and the difference is sizable for some referees. For Salazar and Vaughn, it amounts to .7-.8  card per match. Given the costs of yellow to a team (both in the match and because of match bans), this can make a difference.

Another way to look for evidence of biased refereeing is to look at the connection between fouls and yellow cards. Given the bigger amount of discretion involved in awarding a yellow card compared to awarding a free kick, do we see some referees get out the yellow more quickly, given a certain number of fouls in a match? For the league as a whole, there should be a positive relationship between the number of fouls in a match and the number of yellow cards. And this is exactly what the data show, as the next graph of the correlation between fouls and yellow cards in MLS for the season as a whole demonstrates.

The graph shows that teams received about 1 yellow for five fouls committed and two yellows for 15 fouls. The next question, of course, is whether this league "standard" is applied similarly by referees across the board or whether the connection between fouls and yellow cards is stronger for some. To see if this is the case, below are the correlations between fouls and yellow cards for each individual referee for the 2010 season.

The first thing to notice is that the slope is positive; for each referee save one, more fouls go with more yellows. That's as it should be - anything else would be strange (as it is in the case of Jorge Gonzalez, for whom the correlation is basically zero at -.015 - yes that's a minus). But what we can also see is that there is considerable variation across referees in terms of how they mete out the additional punishment of a yellow card per number of fouls in a match. For some referees like Paul Ward or Terry Vaughn, there is a strong positive relationship; more fouls will earn you more yellow cards when these gentlemen are in charge. This is also the case for Edvin Jurisevic, Jason Anno, Michael Kennedy, Ramon Hernandez, and Mark Geiger. But for others, there isn't much of a connection at all. Good examples are Abbey Okulaja, Jair Marrufo, Kevin Stott, or Silviu Petrescu.

So what do we make of this all? These data can't answer the questions of whether MLS refs are bad or biased - they are not really designed to do that - but there are some noteworthy patterns. First, MLS referees do not call as many fouls as referees in the European leagues. I suspect that this is not because there are fewer fouls in MLS, but because the referees either consciously (because they are told by the league) or unconciously (because they are less competent) call fewer fouls. Second, there is relatively little variation across referees in the number of fouls called per match (suggesting either that the quality of referees is identical or that they are following instructions). Third, there are some differences in referees' decisions to award yellow cards to home and away teams, with away teams punished at slightly higher rate. To me, this looks like evidence of differences across individual referees in making discretionary decisions. Fourth, perhaps the strongest evidence of differences in how referees exercise discretion is in the connection between fouls and cards: clearly, for referees like Stott or Gonzalez, it doesn't really matter how many fouls a team commits - the number of yellow cards doesn't go up. In contrast, there is much more corresponding yellow card punishment (at the team level of course) for referees like Ward or Jurisevic. 

So does David Beckham have a case? On average, Abbey Okulaja called among the fewest fouls in MLS last season, he doesn't give more yellow cards to away teams, and he has one of the lowest correlations among referees between fouls and yellow cards (.19). But he does call more fouls on away teams. In the match between DC United and the Galaxy, he was true to form, calling 10 fouls on DC United and 12 on the Galaxy (hardly a difference), and 2 yellows and 1 red for each team. But of course, one call can make all the difference in a game of rare events - and that's the nature of the beast we love.