Tuesday, August 26, 2008

Does "Clutch" Exist in Quarterbacks?

It has been the subject of debate for a long time- Does clutch actually exist, or is it a figment of our imagination? Famous baseball stats guru Bill James is famous for asserting that clutch hitting is all fiction. The statistics, however, have not been so completely supportive of that statement.

One thing is certain, and that is that people often see "clutch" and "choke" performers where they do not exist. Alex Rodriguez, for instance, has borne the reputation of a choke artist, even though altogether, his late-inning, close-game numbers are just as good as his overall numbers.

First of all, how do we even define, Clutch? Well, I felt that a pretty apt definition of a clutch situation is the 4th Quarter, with the two teams within 7 points of each other. These are the late-game, high-pressure situations. Only quarterbacks are being tested. The test as to whether a quarterback is clutch is whether his stats are statistically so different from his normal stats that random chance can be ruled out. The stat being used is quarterback rating. As discussed in an earlier post, QB rating has gotten unfairly criticized bvut is actually a very valid measure of QB performance.

Ok, with the tests done, guess what?...Clutch does exist. Whereas the majority of quarterbacks play at the same level in normal situations as they do in clutch situations, there are 11 quarterbacks who are dramatically different players in crunch time.

Not only that, but clutch performance seems to be a persistent trait. For the players affected by clutch, I went back at previous years of their careers and found that clutch situations consistently produced different results in them than normal situations. Another, somewhat surprising result- Vulnerability to clutch situations does not appear to be something that guys get over. Those who had it early on seem to always play differently in the clutch.

Before we get to the players affected by clutch situations, a word about the others. Most quarterbacks, it turns out, are not affected by clutch situations. Peyton Manning, Tom Brady, and Tony Romo, to name three, play at a very high level in both normal situations and clutch situations. On the, uh, other end of the spectrum, the same is usually found. Most mediocre quarterbacks remain that way in the clutch. They don't become good, nor do they shrink and become terrible.

Ok, now for the quarterbacks who play differently in clutch situations. Of the eleven, 8 of the quarterbacks play significantly worse in the clutch than they do normally. The other 3 quarterbacks actually rise to the occasion and play substantially better in the clutch than they do normally.

So, without further ado, these are the three quarterbacks who perform better in the clutch than they do normally:

------------------------Overall QB Rating Clutch QB Rating
Matt Hasselbeck 91.4 -------------99.9
Kurt Warner 89.8-----------------------96.5
Sage Rosenfels 84.8-----------------------116.9


I know, right? Sage rosenfels, who woulda thunk it? Now, I know what some of you are thinking. Rosenfels only played half a year, and these numbers are just a fluke. Well, not necessarily. His sample size is smaller than others, but it is still large enough to draw a statistical conclusion, given the disparity between the two QB ratings. Also, a review of Rosenfels's (admittedly brief career) demonstrate that year after year, he performs significantly better in clutch situations than he does overall. He has done this for so many years that, even though the sample size is always modest, it's difficult to ignore.

As for Hasselbeck, he is excellent virtually all the time and becomes even better in the clutch. This pattern, too, has repeated itself throughout Hasselbeck's career. Warner had an underappreciated splendid season last year. His one real bugaboo, which Qb rating does not cover, is a propensity to fumble.

So, who would I want as my quarterback with the game on the line? Tom Brady or Peyton Manning. as I noted earlier, most quarterbacks are the same in the clutch as they are normally. Manning and Brady are two such quarterbacks. In the clutch, they are their normal brilliant selves and are still better than even Hasselbeck, who rises to the occasion.

Below are the eight quarterbacks who, for whatever reason, become shells of themselves when the score is close late in the game:


---------------------------Overall QB Rating Clutch QB Rating
Brett Favre 95.7----------------82.0
*Drew Brees 89.4-----------------------76.8
Jay Cutler 88.1-------------------68.7
Carson Palmer 86.7--------------------63.7
Chad Pennington 86.1--------------------58.3
Jon Kitna 80.9--------------------73.8
Jason Campbell 77.6--------------------66.6
Marc Bulger 70.3--------------------35.6


That last number is not a typo. Bulger's numbers in clutch situations were that bad this year. He has not always been as bad a clutch performer, but a review of his career demonstrates that he consistently performs worse in the clutch than he does normally.

Cutler and Campbell have less than two full seasons under their belt, so that have slightly more chance to change. I would not hold my breath though. Statistical analysis demonstrates that cllutch performance cannot really be learned by the time you get to the pros. By then, it's either in you or it's not.

Carson Palmer had one anomalous season a few years ago, in which his clutch performance was significantly better than his normal performance. However, that seemed to be one exception in a career that has seen consistenly bad clutch performance.

Drew Brees is really the enigma of the bunch. As I have said, clutch performance is normally an enduring trait, and everybody else on this list has demonstrated consistency in their clutch performance throughout their career. Brees is the exception to this rule, which is why I'm giving him the asterisk. Some years, he's fine, and in some years, he's even better in the clutch. Oh well, not even science can explain everything. For all other players on this list, however, their 2007 numbers are indicative of a career-long trend.

I am also aware that over the past decade or so, the fastest growing religion in the United States has been the Worshippers of Favre. I am not saying that he has not had a great career or that his records are not impressive. all I'm saying is that, 1. he's overrated, and 2. His legion of followers, especially in the media, have created a myth that when the chips are down, there is nobody like Favre. This belief is patently false.

Not only has Favre been less than stellar throughout his career in the clutch, but his clutch numbers are actually pretty woeful compared with other situations. i'm not even talking about his well-documented flops in the postseason, like the terrible game he played in this year's NFC championship, getting shut out of the endzone at home against the Falcons, or blowing a divisional game multiple times against the Eagles.

In Brett Favre's career, his clutch performance over a season has only been as good as his overall performance three times. Every other year it was worse. A whopping 10 times, the difference between his clutch numbers and his real numbers was significantly different, statistically speaking. Even in his glorious Super Bowl-winning year, his qB rating in the clutch was an unbelievable 21 points lower than his normal QB rating. seven times, his clutch QB rating was more than 15 points lower than his normal rating.

It is time to put the myth to rest. Brett Favre is not clutch. He never has been. Yes, he may have had some great late-game performance, but as the numbers demonstrate, those were the exception, not the rule. Ok, I think I've pre-empted the Favre defenses, but feel free to let loos anyway.

so now, the truth is out there. Clutch exists, and not everybody has it.

4 comments:

Brian Burke said...

Interesting topic. A couple quick questions--Where did you get your data? And how many seasons of data did you use to define a QB's baseline non-clutch performance level?

My only concern is that "clutch" situations comprise a very small portion of any QB's pass attempts. I wonder if you took a similarly-sized random cross-section of performance if you'd see the same pattern, i.e. similar levels of over- and under-performance in the sub-sample.

Plus, if you're looking at say 20-40 QBs, you'd probably find statistically significant "clutchness" in a couple just due to randomness.

The Football Know-it-all said...

The data comes stratight from the websites of ESPN and NFL.com.

Clutch situations often do comprise a small portion of QB attempts, but the samples of the ones reported on were large enough to make a statistically significant findings. For instance, with the appropriate confidence interval, a sample of 110 passes can be large enough to tell whether it statistically deviates from the population of 500 attempts.

As I mentioned in the post, the numbers used are for the 2007 season only, but in order to double check, I reviewed the others years for any quarterbacks found to have a significant difference between their baseline performance and clutch performance. What I found in the vast majority was a recurring theme, with year after year demonstrating the same statistical pattern.

Brian Burke said...

Could you provide the link to the data? I can't find QB rating splits according to what you describe (by quarter/by score difference) on either nfl.com or espn.com.

Can over 20% of any QB's passes really be in clutch situations (110/500)?

Sorry to pester. I like this topic.

The Football Know-it-all said...

http://sports.espn.go.com/nfl/players/splits?playerId=5536&sYear=2006

Above is the link to Ben Roethlisberger's 2006 splits. If you follow it, you should be able to get other QB's numbers as well. Let me know if there are still any problems.

P.S. The numbers vary quite a bit. In Roethlisberger's 2006, for instance, he had 469 attempts, an amazing 152 of them in clutch situations.