DawgTalkers.net Forums DawgTalk Pure Football Forum Is ESPN's Total QBR a good stat?

Quote:

---

Early in a scoreless game, a quarterback throws a 20-yard pass just by the reaching arms of a defender and into the hands of his intended receiver, who holds on despite the distraction, then scampers the remaining 15 yards for a touchdown.

Another quarterback, down 30-10 with five minutes left in the fourth quarter, throws a 3-yard screen pass to a running back, who maneuvers another 32 yards through prevent defense to pick up a first down deep in opponent territory.

Both are called good plays, but labeling them as "good" isn't enough. Each play has a different level of contribution to winning, and each play illustrates a different level of quarterback contribution. What is the quarterback's contribution to winning in each situation? Coaches want to know this; players want to know this; and fans want to know this.

The Total Quarterback Rating is a statistical measure that incorporates the contexts and details of those throws and what they mean for wins. It's built from the team level down to the quarterback, where we understand first what each play means to the team, then give credit to the quarterback for what happened on that play based on what he contributed.

At the team level, identifying what wins games is not revolutionary: scoring points and not allowing points. Back in the 1980s, "The Hidden Game of Football" did some pioneering work on that topic and on how yardage relates to points. We went back and updated what that book did … then we went further. At the individual level, more detailed information about what quarterbacks do is really necessary. Brian Burke at AdvancedNFLStats.com has done very good work in advancing that effort, and FootballOutsiders.com has done some of this by charting data, but, for the past three years, ESPN has charted football games in immense detail. By putting all these ideas together and incorporating division of credit, we have built a metric of quarterback value, the Total Quarterback Rating, Total QBR or QBR for short.

What follows is a summary of what goes into QBR. It took several thousand lines of code to implement, but we'll keep this shorter.

Win Probability and Expected Points
The goal behind any player rating should be determining how much a player contributes to a win. We went back through 10 years of NFL play-by-play data to look at game situation (down, distance, yard line, clock time, timeouts, home field, field surface and score), along with the ultimate outcome of the game, to develop a win probability function.

Total QBR Basics

A quick primer on the fundamentals of Total Quarterback Rating:

Scoring: 0-100, from low to high. An average QB would be at 50.
Win Probability: All QB plays are scored based on how much they contribute to a win. By determining expected point totals for almost any situation, Total QBR is able to apply points to a quarterback based on every type of play he would be involved in.
Dividing Credit: Total QBR factors in such things as overthrows, underthrows, yards after the catch and more to accurately determine how much a QB contributes to each play.
Clutch Index: How critical a certain play is based on when it happens in a game is factored into the score.

This function treats every win the same, regardless of whether it was 45-3 or 24-23, though there is clearly a difference between such games. The first game represents total domination, whereas the other represents two fairly evenly matched teams. Because win probability treats every win the same, it misses some of what goes into the win, specifically many of the points that represent domination or the points that lead up to a last-second victory. So, although QBR uses win probability to assess how "clutch" a situation is, it uses expected points as the basis of evaluating quarterbacks. It has more of the details, and understands the difference between wins, but still strongly relates to wins in general.

The concept of expected points was discussed as early as the mid-1980s with Pete Palmer & Co. and "The Hidden Game of Football," in which they talk about "point potential." Their idea was that, as you move closer to the opponents' end zone, you are actually gaining points. Brian Burke took it further to note that third-and-10 from midfield, for instance, has fewer expected points than first-and-10 from midfield. In other words, down and distance also matter in terms of points. We took this even further to look at clock time, home field, timeouts and field surface to generate the expected points for any team given its situation in a drive. One particular situation to note is that, at the end of the half, a team is less likely to score any points than at most other times of the game, just because the half is going to expire.

It's useful to mention here that expected points are expected net points. It's possible that a team has expected points less than 0. This simply implies that the other team is generally more likely to score. This usually happens when a team is backed up deep in its own side of the field, especially if it is third or fourth down.

What then happens is an evaluation of expected points added. How does a team go from 1.1 expected points to 2.1? However it does it, that is 1.0 expected points to be distributed to the offensive players on the field. But how the team does it is what determines how credit is given to a quarterback.

Dividing Credit
Division of credit is the next step. Dividing credit among teammates is one of the most difficult but important aspects of sports. Teammates rely upon each other and, as the cliché goes, a team might not be the sum of its parts. By dividing credit, we are forcing the parts to sum up to the team, understanding the limitations but knowing that it is the best way statistically for the rating.

On a pass play, for instance, there are a few basic components:
• The pass protection
• The throw
• The catch
• The run after the catch

In the first segment, the blockers and the quarterback have responsibility for keeping the play alive, and the receivers have to get open for a QB to avoid a sack or having to throw the ball away. On the throw itself, a quarterback has to throw an accurate ball to the intended receiver. Certain receivers might run better or worse routes, so the ability of a QB to be on target also relates somewhat to the receivers. For the catch, it might be a very easy one where the QB laid it in right in stride and no defenders were there to distract the receiver. Or it could be that the QB threaded a needle and defenders absolutely hammered the receiver as he caught the ball, making it difficult to hold on. So even the catch is about both the receiver and the QB. Finally, the run after the catch depends on whether a QB hit the receiver in stride beyond the defense and on the ability of a receiver to be elusive. Whatever credit we give to the blockers, receivers and quarterback in these situations is designed to sum to the team expected points added.

The ESPN video tracking has been useful in helping to separate credit in plays like these. We track overthrows, underthrows, dropped passes, defended passes and yards after the catch. The big part was taking this information and analyzing how much of it was related to the QB, the receivers and the blockers. Not surprisingly, pass protection is related mostly to the QB and the offensive line, but yards after the catch is more about what the receiver does. Statistical analysis was able to show this, and we divided credit based on those things.

As a relevant side note, statistical analysis showed that what we call a dropped pass was not all a receiver's fault, either. A receiver might drop a ball because he wanted to run before catching it, because the defense distracted him, because it was a little bit behind him or because he was about to get hit by a defender. If the defender was there a half second before, the defender would have knocked the ball free and it would have been called a "defended pass," not a "dropped pass." There are shades of gray even on a dropped pass, and analysis showed that. Drops are less a QB's fault than defended passes or underthrows, but the QB does share some blame.

On most other plays, quarterbacks receive some portion of credit for the result of the play, including defensive pass interference, intentional grounding, scrambles, sacks, fumbles, fumble recoveries (Carson Palmer once recovered a teammate's fumble that saved the game for the Bengals) and throwaways.

On plays when the QB just hands off to a running back, we didn't assign any credit to the QB. Our NFL experts did suggest that some QBs are very good at interpreting defenses pre-snap and identifying better holes for their backs. However, they also told us it would be nearly impossible to incorporate. Because they suggested this, we built in the ability to give credit for QBs when they just handed off, but we couldn't find the right analysis to do it in 2011.

Clutch Index
The final major step is to look at how "clutch" the situation was when creating expected points. A normal play has a clutch index of 1.0. For instance, first-and-goal from the 10-yard line in a tie game at the start of the second quarter has a clutch index of almost exactly 1.0. A more clutch situation, one late in the game when the game is close -- the same situation as above but midway through the fourth quarter, for example -- has a clutch index of about 2.0. Maximum clutch indices are about 3.0, and minimum indices are about 0.3.

These clutch index values came from an analysis of how different situations affect a game's win probability on average. One way to think of it is in terms of pressure. A clutch play is defined before the play by how close the game appears to be. Down four points with three seconds to go and facing third-and-goal from the 3-yard line -- that is a high-pressure and high-clutch index situation because the play can realistically raise the odds of winning to almost 100 percent or bring them down from about 40 percent to almost zero percent. The same situation from midfield isn't as high pressure because it's very unlikely that the team will pull out the victory. Sure, a Hail Mary can pull the game out, but if it doesn't work, the team didn't fail on that play so much as it failed before then. On third-and-goal from the 3-yard line, failure means people will be talking about that final play and what went wrong.

The clutch indices are multiplied by the quarterback's expected points on plays when the QB had a significant contribution, then divided by the sum of the clutch indices and multiplied by 100 to get a clutch-valued expected points added per 100 plays.

A Rating from 0 to 100
The final step is transforming the clutch-valued expected points rate to a number from 0 to 100. This is just a mathematical formula with no significance other than to make it easier to communicate. A value of 90 and above sounds good whether you're talking about a season, a game or just third-and-long situations; a value of four or 14 doesn't sound very good; a value of 50 is average, and that is what QBR generates for an average performance.

That being said, the top values in a season tend to be about 75 and above, whereas the top values in a game are in the upper 90s. Aaron Rodgers might have gone 31-of-36 for 366 yards, with three passing TDs, another TD running, 19 first-down conversions, and eight conversions on third or fourth down in one game -- for a single-game Total QBR of 97.2 -- but he can't keep that up all year long. Pro Bowl-level performance for a season usually means a QBR of at least 65 or 70. We don't expect to see a season with a QBR in the 90s.

Defensive Adjustment
With this rating, we have intentionally not adjusted for opponents. This doesn't mean that we won't adjust for opponents as we use it but that we want QBR to be flexible for many purposes, and keeping opponents' strength out gives us that flexibility. As it stands, QBR can be broken down for all sorts of situations -- red zone, third-and-long, throwing to a certain receiver, in bad weather, against different defensive formations. We didn't want to muddy it up with opponent adjustments that aren't as useful for those situations. How to implement a defensive adjustment for third-and-long also might be different from one for the whole season. Beyond this, a defensive adjustment is often not a constant factor. A defense that looks good in Week 4 might not be as good after a few more weeks. Because it isn't a constant thing, it makes sense to leave that for analysis rather than constant incorporation into QBR.

There will be analyses that we do on ESPN that will suggest the use of an opponent adjustment, but we will do that when needed, not up front.

Concluding Thoughts
What underlies QBR is an understanding of how football works and a lot of detailed situational data. What it yields are results that should reflect that. It illustrates that converting on third-and-long is important to a quarterback. It shows that a pass that is in the air for 40 yards is more reflective of a quarterback than a pass that is in the air for 5 yards and the receiver has 35 yards of run after the catch. These premises should sound reasonable to football fans. They come out of a lot of statistical analysis, but they are also consistent with what coaches and players understand.

As we neared the end of the development of QBR, we talked to Ron Jaworski and Greg Cosell at NFL Films about its evolution. Cosell said at one point, "Football is not complex, but it is very detailed." I realized then that QBR is like that. It is very detailed, accounting for a lot of different situations, but it is not particularly complex. It really does try to see the game the way we have gotten used to seeing it in its elegant simplicity. We hope you, the fan, appreciate it, as well.

---

Quote:

---

Q. Is it subjective?
A. There is a perception, mainly from people who watched the QBR special, that QBR has a lot of subjective components. Some who watched the show thought that the expected points were allocated subjectively by people watching games -- give this guy a point or take away a point. This is not true and something we need to clear up.

What underlies QBR is expected points and win probability. In determining expected points and win probability as they relate to field position, down, and distance, there is no subjectivity other than slight differences in how these models are built (which is why AdvancedNFLstats.com doesn't have exactly the same numbers we have). These differences are definitely small.

The part of QBR that could be cynically called "subjective" is that there are judgment calls with regard to what are dropped passes vs overthrows or underthrows or defended passes. ESPN's video trackers have strict guidelines on how to chart these items so that they are consistent across the different people doing charting. If you as a fan go out and chart these yourself for a game or two, you will see how several calls are easy, but some are quite hard to judge. We have standards that make things more uniform and every game is done twice to reconcile inconsistencies. Despite the standards, the gray areas will still exist and, because they exist, the division of credit quantitative analysis described below is important. That analysis is what says that a "drop" isn't necessarily all about a receiver because there are gray areas in drops.

Notably, the kind of judgment calls here are not unique. Every week, statistics like hurries, tackles, or targets get used but have similar judgment necessary to decide them. Neither of these are official NFL statistics and both come with clear gray area. Coaches are known to spend hours going back to evaluate credit on various plays. Our hope is that any statistics used to evaluate individuals in football come with analysis to help split the credit in these more gray areas. We did that analysis to limit subjectivity.

Q. Is it a black box?
A. One of the main concerns that people have had with Total QBR is that it is a "black box," spitting out results without saying how those results are generated.

It is true that there isn't a simple formula to just calculate QBR, but we wrote the QBR Guide and are writing this to make clear that it is not meant to be a statistic where we say, "Trust us." ESPN The Magazine's NFL Preview edition also explains aspects of this to be more transparent.

The method we followed to generate QBR is another step in the line of work done in Hidden Game of Football, at AdvancedNFLStats.com, and at FootballOutsiders.com. The methodology for Football Outsiders' Adjusted Line Yards is the same one used for dividing credit in QBR. The basic premise behind AdvancedNFLStats' work on expected points and expected wins is the same for QBR; we did it a different way and accounted for a few more factors, but a lot of results are essentially the same. If you know the work at those sites, the work behind QBR is similar.

Will it ever be calculable with a calculator? Not easily because it does look at every play. But nor is the expected points of AdvancedNFLstats easily calculated. They have an online tool to generate expected points and win probability; those and other tools will eventually come to ESPN, as well.

Will the components of QBR be available? Yes. We can look at how much interceptions, sacks, and fumbles hurt a QB and we intend to display that on ESPN.com. We can look at completions and scrambles and designed rushes and see how much those added to a QB's rating. These will be available online so that people can see where players fit. A representation of the QB's average clutch index will also be shown. These elements will add to the story and putting those out there will help fans see what QBR sees.

Q. How does the division of credit work?
A. Division of credit for a play is done a lot in Total QBR. Pass plays are the result primarily of an offensive line giving a QB time, a QB making a good decision and throwing accurately, and a receiver holding on to the ball and turning it into as many yards as he can. In most other work, all of the yards the team gets is given to the quarterback and the receiver (and the offensive line, if it ever got talked about), but this really double-counts those yards. We know that the credit really should be split and the analysis to split the credit has been available for sports for a while.

Specifically, Ben Alamar did this for the Adjusted Line Yards work of FootballOutsiders, where he looked at what percentage of yards are the offensive line for long and short gains. He also did it for a paper in baseball on whether the pitcher or the hitter controls the plate in baseball. From that work, we brought him in to do the same thing here.

The details of how it works are spelled out in the paper on baseball, but we will attempt to describe it conceptually here.

Let's use the example of a quarterback throwing to a receiver. If you have a quarterback who throws a lot of passes that get dropped and a receiver who doesn't drop a lot of passes, what will happen when that quarterback throws to that receiver? Intuitively, we think of dropped passes as mostly associated with receivers, so the QB with a lot of drops would probably be labeled unlucky and the receiver would have "good hands." So the pass is more likely to be complete and not dropped. That is intuition.

The analysis looks at the mathematical way to predict whether the ball will be dropped and whether the factors that make that prediction are more quarterback-related or more receiver-related. If the factors are more receiver-related, then drops are more on the receiver. If the factors are more associated with the QB stats, then drops are more on the quarterback. In the end, analysis does generally support intuition -- drops are more on the receiver.

So it would predict that the pass wouldn't be dropped and the QB needs more receivers like this one who don't drop the ball.

The benefit of doing this is that it uses the existing data from our video tracking team to assess the division of credit. So, given the way they have charted drops and overthrows, for example, this method suggests how much is the QB and how much is the receiver. There are some gray areas in charting drops, overthrows, etc. where a drop could be an overthrow or an overthrow could be a drop. By doing this analysis, it accounts for that gray area and the division of credit at the same time.

Q. How come quarterbacks get so much blame for sacks?
A. One of the earliest conversations we had when we took on the task of developing QBR was about sacks. We had heard from Trent Dilfer and others that many sacks are on the quarterback, not the offensive line. Some quarterbacks just hold the ball too long because they are indecisive or not confident in their ability. Pro-football-reference.com did a study suggesting that quarterbacks are "more responsible for sack rate than we believe." Anecdotally, when we looked at Matt Cassel taking over for Tom Brady in New England in 2008, Brady's sack percentage was 3.5% in 2007 and 2.8% in 2009, but Cassel was sacked 8.3% of the time in 2008, which is closer to Cassel's career sack percentage of 7.4% than the sack percentage of the New England offensive line over that period.

Dividing credit on this factor is difficult because the line and the quarterback are often tightly linked, with quarterbacks rarely changing and their offensive line also not changing much. What was available were things like the number of rushers and, for a select group of games, the amount of time a quarterback had to throw. Intuitively, one would think that the more time a quarterback had before they got sacked, the more responsible the quarterback (and their receivers) is for taking a sack -- the line can only hold on for so long. This is not always true, but it is a rule of thumb.

Similarly, with more rushers, there is less that the offensive line can do to keep them from the quarterback. The quarterback can better see and adapt to extra rushers by finding receivers with single or no coverage. The analysis we did with the data supported both these ideas. That analysis also suggested that, on average, a little more than half of the blame for sacks is on the quarterback. With extra rushers, it goes up to about 60% on the quarterback. The data for time to throw the ball was more sparse and thus less conclusive, but suggested about 53% of blame for sacks goes to quarterbacks on average.

These are average results and there are variations in offensive lines, but it is important to see that the analysis suggests half of sacks are typically on the QB. One of the long term improvements we do envision is more refined data to split credit between quarterbacks and their blockers for sacks and other plays.

Q. Explain the Clutch Index.
A. The Clutch Index or Clutch Weight or, for those familiar with baseball analytics, the Leverage Index is a measure of how important any play is towards changing the winning percentage of the game. This Clutch Index is calculated pre-play and does not depend on the outcome of the play. We think of it as generally reflecting how much pressure a player may feel on the play. For example, imagine a team down 4 with 3 seconds to go in the game. In one case, it is 3rd and goal from the 3 yard line. In the second case, it is 3rd and 10 from midfield. In the first case, it is a high pressure situation. The team can win with the right play call, the right block, or the right pass, but they can also lose the game with the wrong call, block, or pass. Whether they win or lose on that last play, people will be talking about the last play after the game. In the second case, it is unlikely a Hail Mary works, so if they don't win on that last play, people will be talking about the rest of the game, not the failure on the last play. Only if they win will people be talking about that last play; there was less pressure then. The Clutch Index reflects this by assigning a much higher value in the first case than in the second.

One of the big things a Clutch Index was set up to do was to minimize the value of plays made in games already decided. When a quarterback is piling up yards in the 4th quarter down several scores, they are doing it against a defense that is not working as hard as they would normally. The defense may be in prevent mode, they may take out their better players, but they just aren't competing the same as you would see in a tight game. Our advisors, from Ron Jaworski to Trent Dilfer, felt strongly that this should be accounted for.

To develop the Clutch Index, we looked at the game time and the game closeness (defined to be how close the win probability was to 50%) and how plays affected the win probability on average. The Clutch Index then became a calculated function just of time in the game and how close it was. Late close games were more clutch than early close games. But early close games were more clutch than a lot of other situations. Blowouts at any time of the game received fairly low clutch indices. This generally did what our advisors asked for, also rewarding performance in tight games.

For those aware of AdvancedNFLStats, that site has WPA, which stands for win probability added. WPA looks at the actual change in winning percentage with every play. It is an intuitively nice concept, but it has strange consequences. Single plays can completely dominate the rest of the game. A game decided on the last play gives almost all weight to that one play, even though the other plays building up to it were important in putting the team in a position to win on that last play. WPA also has the flaw of weighting every win the same amount. A 45-3 win is not viewed any differently than a 24-20 win even though there is a big difference in those games in that one represented domination and the other could be luck. A quarterback in that first game probably played great, whereas in that second game, the QB may have played great, but threw an incompletion on 3rd and goal from the 3 with 3 seconds left to "lose" the game. WPA for that second QB would probably reflect too much of that last play and not enough of the whole game.

Finally, a question we got from Aaron Schatz at Football Outsiders was this, "If you have two QBs with the exact same performance, but one has a bad defense and the other has a good defense, will the QB with the bad defense get a better rating?" The question is motivated by the idea that a good quarterback with a bad defense will be facing more close games than a good quarterback with a good defense. As a result, their clutch opportunities will be higher. The answer to the question is actually not clear. Because QBR normalizes by how many clutch opportunities quarterbacks get, there is no straightforward answer to the question. If they are good quarterbacks and both do exactly the same in clutch situations vs non-clutch situations, then they will both have the same value for QBR. That is as straightforward an answer as we can give.

Q. Why doesn't it have a defensive adjustment?
A. In the Guide to Total QBR, we talk about why there is no adjustment for the opponent played. There are many reasons, though. First, an opponent adjustment really is variable. We don't know how good an opponent is for sure. The adjustment for opponent can change over the course of the season, which causes problems interpreting QBR. If a guy goes into Week 5 with a QBR of 50, then doesn't play because it's a bye week, but the teams he has played against all give up a ton of points, the estimate of the opponent adjustment can change a lot. This would then lower an opponent-adjust QBR value. We didn't want the headline QBR statistics to have that kind of variability that is just associated with estimates of opponent quality.

The basic adjustment for opponents or defense is to look at opposing teams and iteratively estimate how good they have been, but there really are multiple ways to adjust for opponent. There is now information available about what specific players were on the field for specific plays. Should we adjust for times in the game when a team has their best or worst players on the field? This does better account for when a player like Ray Lewis gets hurt, but it then requires a more player-specific defensive evaluation.

Even further, QBR was meant to be able to be sliced to work with all sorts of situations. We can look at QBs in 3rd and long or 3rd and short, for example. The defense they face in 3rd and long is different than the one in 3rd and short. Should there be different defensive adjustments for these two? Probably so. These complexities are what led us to keep opponent or defensive adjustments out of QBR. It complicates the story instead of simplifying it.

This being said, there will be times over the course of a season to talk about opponent adjustments, but we decided that it would be better to do OUTSIDE of Total QBR itself. Doing it outside of it allows it to be added when needed and allows it to be added in different ways depending upon the data that are available or upon the intent of the analysis.

Q. What is the point of a new quarterback rating?
A. First of all, the NFL Passer Rating needed an update. It was meant to rate quarterbacks only as passers, not rushers or scramblers. It wasn't meant to account for fumbles. It didn't consider sacks to be part of what passers do, though there is plenty of evidence now that they are at least partly associated with the QB. It was built in the 1970s to apply to full seasons with less applicability to games or small groups of plays, like third down. As a tool for evaluating quarterbacks for all that they do, the NFL Passer Rating was missing too much.

Upon design, we wanted to create a quarterback rating that could be used as a tool for identifying a better quarterback. If a General Manager is making a decision about quarterbacks, what would he want to consider? We kept that question in mind as we built it, but part of answering that question is not only the What but How. When using a statistical tool to help make a decision, a critical part is the ability to break it down -- understand what goes into it.

Total QBR is very easily broken down. Quarterbacks get expected points through completions and incompletions, interceptions, interception returns, sacks, fumbles, fumble recoveries, scrambles, designed rushes, defensive pass interference penalties. This tool can show how much comes from these different components. If we find that drawing defensive pass interference penalties isn't a repeatable skill, then making a decision about a quarterback can use QBR without the defensive pass interference part. Defensive pass interference was a part of his past performance, so we want it available and in there so that, when we get to ratings for other positions, all the players add to a total, but it can be separated out. QBR gives that flexibility to produce useful ratings, but tell the parts of the story as needed.

Another important design factor for Total QBR was that it be difficult to "game the system." We heard frequently that quarterbacks knew how the NFL Passer Rating worked, so they would try to maximize their Passer Rating even if it didn't help their team win. Total QBR is built upon team success, then broken down to the quarterback contribution, so that a quarterback's incentives and his team goals are better aligned. Racking up lots of yards in a meaningless situation doesn't help a QB in QBR like it did with NFL Passer Rating. Going back again to what an NFL GM would like, aligning a quarterback's statistical performance with that of the team helps make contract negotiation easier.

Q. How could Eli Manning be above average with 25 interceptions in 2010? How did Philip Rivers' interception against Dallas not matter? How come Aaron Rodgers doesn't rank higher with how he played in the playoffs? Was Jay Cutler that bad? How is Matt Ryan or Colt McCoy so high in the rankings?
A. These are various result-related questions. We had to look at many of them as we went through the process to make sure things were being done right. We will address a few of them here.

How could Eli Manning be above average with 25 interceptions in 2010?
When we first saw Eli Manning at #7 in 2011 with 25 interceptions, we started investigating. One of the biggest components of the mismatch in perception and rating is in that Manning had 4 interceptions last year that were after a receiver really should have caught the ball. He had several others that hit the hands of receivers. He had a larger than average number of his incompletions also dropped by receivers, yet he still was in the top 10 in completion percentage. Another component of the mismatch is that he really took few sacks last year. Finally, he was, besides the interceptions, quite productive, throwing for a lot of yards downfield and a lot of touchdowns.

One thing that our advisors suggested about many quarterbacks, but with Eli Manning as the prime example, is that they often are integral to the running game. Quarterbacks with freedom to call audibles make reads at the line of scrimmage and can change which hole the backs go to, really helping certain rushers gain their yards. We looked into this but didn't have data or the right analysis to find this. Nonetheless, we have been told that Giants fans should appreciate this subtle aspect of Eli Manning's game.

How did Philip Rivers' interception against Dallas shown in the QBR Special not matter?
In the show that presented Total QBR to the public, there was a Philip Rivers interception from his own end zone that went out to about his own 30. It was portrayed as less harmful than a David Garrard interception that was returned for a touchdown. It should be clear that it is less harmful, but we want to now make clear that Rivers' interception was not a meaningless interception, which could be construed from the wording on the show. A meaningless interception is the Hail Mary at the end of the half that doesn't get returned for a touchdown. Rivers' interception wasn't as bad as many, but it was a negative play and it was underthrown. We apologize for the confusion.

How come Aaron Rodgers doesn't rank higher with how he played in the playoffs?
Aaron Rodgers played tremendously in the playoffs. It was, by QBR's count, the best playoff run in the last three years with a QBR of 86.5. But his regular season number was 67.9 -- still good, but not other-worldly as he was in the playoffs. We have not emphasized sufficiently that 67.9 is just his regular season number and doesn't incorporate what he did in the playoffs. As with all NFL statistics, we are keeping playoff and regular season numbers separate.

How is Matt Ryan or Colt McCoy so high in the rankings?
These are a couple of the more common quarterbacks whose QBR value is questioned. Matt Ryan ranked 3rd in 2010 and McCoy had a 46.6 QBR as a rookie, higher than the 41.0 for Sam Bradford.

Ryan's performance was, by most metrics, quite good. He was a Pro Bowler, his 91.0 NFL Passer Rating was 6th, the Falcons were in the top 10 in team offensive EPA, and the team won 13 games. He played poorly in his playoff matchup against Aaron Rodgers, with a QBR of 11, so the last impression people have of him is a poor one, so perhaps that is part of the mismatch. The legend of "Matty Ice," that he is cool under pressure, does show up in QBR calculations. His QBR in 3rd and long was 78 and 68 in the 4th quarter of close games. Using AdvancedNFLStats and their WPA calculation for QBs, which strongly reflects performance in clutch situations, Ryan was the best QB in 2010.

Colt McCoy was 2-6 as a starter for the Cleveland Browns as a rookie. Sam Bradford drew so much more attention for his rookie year with the Rams that McCoy didn't get noticed. With just 6 TDs in those 8 games, McCoy also wasn't the guy putting the ball in the end zone. And, like Ryan, his last impressions weren't very good, throwing 6 interceptions in his last 70 attempts after throwing 3 in his first 150. But that is part of the story -- he was pretty good out of the gate, with 5 of his first 6 games having an NFL Passer Rating over 80. The Browns beat the Patriots and Saints and had 3 losses by less than one score. In the Browns' 8 games with McCoy as starting QB, they had -10 expected points added as a team; in the other 8, they had -54. And, if we want to do some opponent adjustment, he faced the Steelers twice, the Ravens, and the Jets -- all top 10 teams in defensive expected points added.

Q. Does QBR relate to winning?
A. How QBR "relates to winning" is a question that can be interpreted various ways. One way that we addressed this on TV was that the team winning the QBR battle within a game wins the game 86% of the time. Teams that win the turnover battle don't win this often. Teams that win the NFL Passer Rating battle don't win this often. This result is primarily emphasizing that QBR is capturing team results and that quarterbacks performance is quite important to that.

But QBR is not meant to be a perfect indicator. A quarterback affects the offense, so a quarterback with a good defense doesn't have to be as good. What illustrates this are the 2008 seasons of Ben Roethlisberger of the Pittsburgh Steelers and Dan Orlovsky of the defenseless Detroit Lions. Orlovsky had a Total QBR value of 51, whereas Roethlisberger posted just a 46 (but a 61 in the playoffs that people remember).

In that year, expected points allowed by team defense had the Steelers second in the NFL and the Lions dead last, with more than 300 points difference on the defensive side of the ball. With that kind of defensive difference, it was possible for Orlovsky to have a better rating than Roethlisberger and still lose a lot more. Orlovsky wasn't terrible either. In the 7 games that Orlovsky started for the Lions, the team offense was about average by expected points added; in the other 9 games, they were roughly 10 points per game worse. In the games started by Roethlisberger, the Pittsburgh team offense was about the same as the Orlovsky-led Lions offense.

So QBR is meant to correlate to offense, moving the ball downfield, turning good field position into points, avoiding giving it back to the defense. Good offensive performance is not the same as winning. To the degree that offense correlates to winning, QBR should be helpful.

---

Quote:

---

No, it is not.

---

Quote:

---

Abridged version:

Quote:

---

No, it is not.

---

---

Quote:

---

Weeden is ranked 31st. Tough as he is still getting drug down statistically from that week1 game, but that is what happens with a game that brutally bad.

---

Quote:

---

Quote:

---

Weeden is ranked 31st. Tough as he is still getting drug down statistically from that week1 game, but that is what happens with a game that brutally bad.

---

I have seen this multiple times and wanted to test it.

With that game Weeden's stat line looks like this: 57% completion rate, 6.51 yards per attempt, 1.1 TD's per game, 1.25 INT's per game, 72.3 QB rating, 235 yards per game.

Without that game Weeden's stat line looks like this: 59% completion rate, 6.8 yards per attempt, 1.2 TD's per game, 1 INT per game, 79.5 QB rating, 245 yards per game.

You're right, huge difference.

---

Quote:

---

Those are actually rather significant variances from just one game. That game was BAD.


The .25 decrese in INT's is HUGE. Actually, just looking at the magnitude of the shift in QB rating shows how significant it is.
With that game, he is nearly down at 70. Without it he is nearly up to 80. That is a MASSIVE shift for a single game.

---

Quote:

---

No. And Trent can go Dilfer himself.

---

Quote:

---

With that game Weeden's stat line looks like this: 57% completion rate, 6.51 yards per attempt, 1.1 TD's per game, 1.25 INT's per game, 72.3 QB rating, 235 yards per game.

---

Quote:

---

Quote:

---

With that game Weeden's stat line looks like this: 57% completion rate, 6.51 yards per attempt, 1.1 TD's per game, 1.25 INT's per game, 72.3 QB rating, 235 yards per game.

---

Without the first Bengals game, his statistically best game, his stat line is...

55.8% completion rate, 6.3 yards per attempt, 1 TD per game, 1.36 INT's per game, 68.35 QB rating, 227 yards per game.

Just saying.. if we are allowed to take stuff out of stat lines to prove your point, then what are you really showing?

---

Quote:

---

Please don't bring this argument to this thread. I created this thread to ask fellow posters if a statistic is valid or not.

---

Quote:

---

Quote:

---

Quote:

---

With that game Weeden's stat line looks like this: 57% completion rate, 6.51 yards per attempt, 1.1 TD's per game, 1.25 INT's per game, 72.3 QB rating, 235 yards per game.

---

Without the first Bengals game, his statistically best game, his stat line is...

55.8% completion rate, 6.3 yards per attempt, 1 TD per game, 1.36 INT's per game, 68.35 QB rating, 227 yards per game.

Just saying.. if we are allowed to take stuff out of stat lines to prove your point, then what are you really showing?

---

Back in Week 6 when everyone was still regularly posting updates to the "all the rookie QB's did this" posts, I wondered what the numbers would look like if you threw out the high & low games for each and then re-cacl'd.

So, yes, your numbers paint the picture as worse (mostly because INT's total doesn't change, but the number of games decreases by one). What do we get if we throw out the Eagles and Cinci games, however (the two primary statistical outliers)?

---

Quote:

---

Quote:

---

With that game Weeden's stat line looks like this: 57% completion rate, 6.51 yards per attempt, 1.1 TD's per game, 1.25 INT's per game, 72.3 QB rating, 235 yards per game.

---

Without the first Bengals game, his statistically best game, his stat line is...

55.8% completion rate, 6.3 yards per attempt, 1 TD per game, 1.36 INT's per game, 68.35 QB rating, 227 yards per game.

Just saying.. if we are allowed to take stuff out of stat lines to prove your point, then what are you really showing?

---

Quote:

---

QBR is what it is.....a contrived stat that tries to bring people to ESPN.com

---

Quote:

---

that it was a statistical anomaly.

---

Quote:

---

basically, Weeden had to go through that game to truly recognize NFL windows and coverages after facing nothing but vanilla defenses in the preseason and it should be expected for a rookie.

---

Quote:

---

Quote:

---

No. And Trent can go Dilfer himself.

---

Good answer. Did you even read the articles?

---

Quote:

---

Quote:

---

Quote:

---

No. And Trent can go Dilfer himself.

---

Good answer. Did you even read the articles?

---

Yes.

And no, anything that is subjective is not a good statistic. Subjectivity has its place, and IMO statistics is not it.

---

Quote:

---

Quote:

---

Abridged version:

Quote:

---

No, it is not.

---

---

But it's as valid as the system we have been using for years...

---

Quote:

---

I don't know if you're specifically talking about QBR or stats in general, but any stat becomes subjective to a certain degree. However, when you have ten different ways of measuring or grading QB's, and 8 or 9 of those ways ultimately place the players in the same places, then you have a solid scale of judgement.

I don't fully buy into QBR, in fact I probably buy into it less than pure QB rating. However, I don't fully buy into either system, as neither takes total yards into account, and in today's NFL that matters even more because of QB's that can run. Still, when you want to compare two QB's, the odds are that if you use both systems in conjunction with each other you'll get a greater sense of the value of each guy individually.

---

Quote:

---

Quote:

---

I don't know if you're specifically talking about QBR or stats in general, but any stat becomes subjective to a certain degree. However, when you have ten different ways of measuring or grading QB's, and 8 or 9 of those ways ultimately place the players in the same places, then you have a solid scale of judgement.

I don't fully buy into QBR, in fact I probably buy into it less than pure QB rating. However, I don't fully buy into either system, as neither takes total yards into account, and in today's NFL that matters even more because of QB's that can run. Still, when you want to compare two QB's, the odds are that if you use both systems in conjunction with each other you'll get a greater sense of the value of each guy individually.

---

Stats in general we can post and talk about they are not changable. If a goes 10-17 he completed 58.8 5 of his passes. We can discuss what that means to us but the numbers are there and can be run by anyone willing to do the math.

The QBR plus rating or whatever this espn stuff is however we cant compute because it much like the refs is a personal call by someone somewhere based on what they feel the grade should be.

Real stats as you say can be used or twisted to show something but in the end we can test those stats. A supposed stat that cant be tested I dont think should be considered a stat.

---

Quote:

---

lol, tell us how you really feel about it, troll.

---

HEHE!

Quote:

---

There really isn't any good stat to measure QB play. At best, QB rating, or even some of the more adjusted/fine-tuned stats are just barometers.

Things don't average out after 500 attempts in a year; the sample size is too small and there are tons of confounding variables as well like surrounding cast, defense, scheme, refs (think Russell Wilson TD vs INT and how that affects the stat line). Defensive back drops a wide open INT, how do you rate that (honest question)? Some QBs have a Calvin Johnson or Jimmy Graham to throw to, others not so much. Think playing behind 49ers O-line vs Cardinals O-line..... and on it goes. I will say that I think the QB is more important than any several other players combined but that's another topic lol

The only real way you're going to have a good picture of exactly how a QB (or any other position, really) is playing is to grade them. People make football out to be way more complicated than it really is, but it's a lot of work and unless you have access to a pro personnel department, there really aren't many good public resources out there for it. The only site that I know of that actually does it gets constantly thrown under the bus because their grades don't match up to many peoples' pre-conceived notions on how good players are.

---

Quote:

---

The QBR plus rating or whatever this espn stuff is however we cant compute because it much like the refs is a personal call by someone somewhere based on what they feel the grade should be.

---

Quote:

---

but it's Trent Dilfer and ESPN, therefore it must be horrible.

---

Yes and yes.... but that's beside the point.


The whole point is that it is just as you say - no different than message board drivel and completely subjective and made-up homerism. Look how often we compare two nearly identical players on here, but because someone has a predisposed preference for the one, that one is good while the other's exact same efforts get downgraded.

That is NOT something that you can build a reliable, meaningful, or even useful, rating system off of.
Good grief, what's next.... mac creating a rating system for the political analysts to use?

Quote:

---

no different than message board drivel and completely subjective and made-up homerism. Look how often we compare two nearly identical players on here, but because someone has a predisposed preference for the one, that one is good while the other's exact same efforts get downgraded.

---