How to identify NFL's best QB's

[Falcon Man™]

Linky

The late Don Smith never claimed his passer-rating formula was perfect.

Quite the opposite, in fact.

"Some people call it a quarterback rating system, but that really is not what it is," Smith told me during a 2002 interview. "It’s simply a passing statistic."

I've actually defended Smith's rating system because the quarterbacks with the highest ratings -- Tom Brady, Philip Rivers and Aaron Rodgers led the way last season -- usually are the best quarterbacks. But there's so much more to quarterbacking than passing stats for touchdowns, interceptions, attempts, completions and yardage.

Game situations should count for something, and now they do.

With input from football people, including ESPN analyst Trent Dilfer, our statistical analysts have developed a 100-point ratings scale for quarterbacks taking into account advanced stats, game situations and relevant non-passing stats, including fumbles and sacks, to evaluate quarterbacks far more thoroughly. The methodology is complex -- one of the formula's key algorithms spans some 10,000 lines -- but the resulting "Total Quarterback Rating" (QBR for short) beats the old passer rating in every conceivable fashion. The ratings scale will debut this season.

[+] EnlargeTom Brady

Al Bello/Getty ImagesAccording to an outline for the rating system, Tom Brady would fall in the "top tier" category.

I've been bugging the Stats & Information team for a sneak peak ever since learning former NBA statistical analyst Dean Oliver had joined our production analytics unit and was playing a prominent role in QBR development. Oliver, a Caltech grad with a Ph.D. in statistical applications, revolutionized how NBA teams use advanced statistics. Menlo College professor Ben Alamar, who has consulted with the San Francisco 49ers, is also part of the team.

Our stats team has been using game video to track stats relating to pressure, personnel, formation, game situation and more since 2008. The QBR stat represents a significant leap in harnessing those statistics for something more.

The old formula Smith created treated stats the same regardless of circumstance. A touchdown pass thrown against a prevent defense during a blowout defeat equals one thrown against pressure to win the game. A 5-yard completion on third-and-4 counts the same as a 5-yarder on third-and-15. A critical quarterback scramble, sack or fumble doesn't even factor.

"There is no way to statistically say how effective a guy is under fire," Smith lamented during our 2002 conversation. "None of that can be put into something like this."

Now it can, along with a whole lot more.

The QBR formula takes into account down, distance, field position, time remaining, rushing, passing sacks, fumbles, interceptions, how far each pass travels in the air, from where on the field the ball was thrown, yards after the catch, dropped balls, defensed balls, whether the quarterback was hit, whether he threw away the ball to avoid a sack, whether the pass was thrown accurately, etc. Each play carries "clutch weight" based on its importance to game outcome, as determined by analyzing those 60,000 plays since 2008. The stats adjust for quarterbacks facing an unusually high number of these situations.

"If it is a running clock late in the game, maybe you only get a few yards here or there, that is the right football play to make," Jeff Bennett, senior director of ESPN's production analytics team, said Sunday. "We spent a month learning about ratings to make sure quarterbacks couldn’t game the system, so they're not afraid to throw that deep pass at the end of the first half and risk an interception."

I've seen an outline for the rating system breaking down 2010 quarterbacks into six general categories, from top tier to poor. Precise rating numbers were not yet available. The quarterbacks under consideration broke down as follows:

Top tier: Brady, Peyton Manning, Matt Ryan, Michael Vick, Rodgers and Drew Brees.

Well above average: Josh Freeman, Eli Manning and Philip Rivers.

]Above average: Ben Roethlisberger, Tony Romo, Joe Flacco, Matt Schaub, David Garrard and Kerry Collins.

Around average: Matt Cassel, Ryan Fitzpatrick, Mark Sanchez, Carson Palmer, Colt McCoy, Kyle Orton and Jon Kitna.

Below average: Shaun Hill, Jason Campbell, Jay Cutler, Matt Hasselbeck, Chad Henne, Donovan McNabb, Sam Bradford and Alex Smith.

Poor: Derek Anderon, Brett Favre and Jimmy Clausen.

(Really? )

ESPN plans to enlist several quarterbacks when introducing the stat during an hour-long "SportsCenter" special Aug. 5 at 8 p.m. ET. We'll be referencing the stat on the blogs and elsewhere. Bennett said he's allocating enough manpower to produce ratings on game days, a huge help for those of us analyzing player performances shortly after games.

"We want to reward a good football play," Bennett said.

[SyCo]

lol oh man you're asking for it. RC trolls are going to love this.

[SuperAINTSfan]

lol oh man you're asking for it. RC trolls are going to love this.

Who are these said TROLLS?!!!

Yes im changing the subject!!! lol

[Big Bubba]

Who are these said TROLLS?!!!

Yes im changing the subject!!! lol

[Big Bubba]

The top tier of Qbs there.. if we assumed each one of them had playoff wins under his belt, which one would we be wrong about?

[SaintRay]

Good thing NFL GM's use more than stats when determining a players value...that is one f'ed up list, top to bottom.

[Durrtay]

i think it's funny considering everyone comparing bradford to ryan... and look where he ranks

[koolkat]

This Topic was on TATF and I knew one of you homer would bring it to RC thinking it was a relevent rating system.

Which it is not because by the same criteria Romo would have been consider elite a few years ago.

I ask this question what makes Ryan over Rivers or Rothisberger (sorry if misspelled)?

[pencilpusher. just because]

Congrats! another paper award for the Falcons!!whoo hoo!

[Serge]

This Topic was on TATF and I knew one of you homer would bring it to RC thinking it was a relevent rating system.

Which it is not because by the same criteria Romo would have been consider elite a few years ago.

I ask this question what makes Ryan over Rivers or Rothisberger (sorry if misspelled)?

Regarding this rating system, I would guess that Ryan graded out higher because he doesn't take as many sacks as those two and was a bit more efficient on third downs.

[pencilpusher. just because]

But I gotta say...

"With input from football people, including ESPN analyst Trent Dilfer...."

:lol:

And all this time I was sure you guys thought he was a moron!

[falconidae]

But I gotta say...

"With input from football people, including ESPN analyst Trent Dilfer...."

:lol:

And all this time I was sure you guys thought he was a moron!

He still is, but undoubtedly his input is diluted by 10,000 lines of code :rolleyes:

[koolkat]

Regarding this rating system, I would guess that Ryan graded out higher because he doesn't take as many sacks as those two and was a bit more efficient on third downs.

Romo didn't take alot of sacks a few years ago either and put up simlar or better # then Ryan. We still didn't consider him elite. I said it before without playoff success you are not in the same catagory as Manning, Brady, Brees and Rodgers.

[pencilpusher. just because]

He still is, but undoubtedly his input is diluted by 10,000 lines of code :rolleyes:

The fact that he input at all however is a bit telling. " we need the best QB and statistical minds in the business! anyone got Trent Dilfers number?"

Say that to yourself slowly.

[Serge]

Romo didn't take alot of sacks a few years ago either and put up simlar or better # then Ryan. We still didn't consider him elite. I said it before without playoff success you are not in the same catagory as Manning, Brady, Brees and Rodgers.

Romo had comparable stats to what Ryan had last year for one year in his career, outside of 2009 he's always turned the ball over a lot. If you're asking why Ryan is in the same category as the elite at the position in this rating system, then you can either acknowledge what the rating system takes into consideration or you can spend more of your time bitching about Matt Ryan on the official Atlanta Falcons message board.

[pencilpusher. just because]

Romo didn't take alot of sacks a few years ago either and put up simlar or better # then Ryan. We still didn't consider him elite. I said it before without playoff success you are not in the same catagory as Manning, Brady, Brees and Rodgers.

Romo also has a better career rating than Ryan,and is # 4 ALLTIME in yards per pass attempt. The 3 guys ahead of him are long dead HOFers.

Romo has a career 97 rating against opponents with 12+ wins ( 7 games, 12 td's}and a 92 rating against teams with 9-ll wins ( 16 games, 31 TD's)

So someones gonna have to explain a few things. Not really tho, I am glad they have their paper awards. They have so little otherwise.

[falconidae]

This Topic was on TATF and I knew one of you homer would bring it to RC thinking it was a relevent rating system.

Which it is not because by the same criteria Romo would have been consider elite a few years ago.

I ask this question what makes Ryan over Rivers or Rothisberger (sorry if misspelled)?

Dude, the guys at ESPN are just trying to build a better rating system, one that take in a lot more variables than the current QB rating system.

Why don't you ask the folks that put the system together? Won't do you a bit of good to ask us, we know as much about the criteria as you do.

I'll give a shot though, to ease your mind. Here's how they made the new rating:

The QBR formula takes into account down, distance, field position, time remaining, rushing, passing sacks, fumbles, interceptions, how far each pass travels in the air, from where on the field the ball was thrown, yards after the catch, dropped balls, defensed balls, whether the quarterback was hit, whether he threw away the ball to avoid a sack, whether the pass was thrown accurately, etc. Each play carries "clutch weight" based on its importance to game outcome, as determined by analyzing those 60,000 plays since 2008. The stats adjust for quarterbacks facing an unusually high number of these situations.

I'm guessing two things- one, that this is a regular season ranking based on last year and that the part I bolded is the reason Ryan is ranked so highly.

Ryan ,because the falcons played the last two years in a grind it out ball control offense to keep their young D off the field, has faced, and done well in more critical situations than many QBs- he has a lot more 3rd downs to deal with [see PFF analysis for that breakdown]

Falcons went 13 and 3 last year because Ryan was incredibly successful on 3rd downs and fourth quarter drives. Looks like that's really important in these ratings.

And again, that's my guess, email Sando and ask him.

[falconidae]

Romo also has a better career rating than Ryan,and is # 4 ALLTIME in yards per pass attempt. The 3 guys ahead of him are long dead HOFers.

Romo has a career 97 rating against opponents with 12+ wins ( 7 games, 12 td's}and a 92 rating against teams with 9-ll wins ( 16 games, 31 TD's)

So someones gonna have to explain a few things. Not really tho, I am glad they have their paper awards. They have so little otherwise.

Agree that Romo is a good QB. Only thing I can figure out is that these ratings are based on last year and missing that many games is figured into the rating.

[pencilpusher. just because]

Romo had comparable stats to what Ryan had last year for one year in his career, outside of 2009 he's always turned the ball over a lot. If you're asking why Ryan is in the same category as the elite at the position in this rating system, then you can either acknowledge what the rating system takes into consideration or you can spend more of your time bitching about Matt Ryan on the official Atlanta Falcons message board.

Romo has a TD/int ratio of 2-1, just like Ryan, and in 15 more career starts has 50 more TD's and 6,600 more yards.It's a given when you throw the ball deep more often, you will get more TD's AND int's. But its still 2-1 ratio and throwing the ball deeper is obviously harder than screen passes and 10 yard routes.

True Romo used to fumble a lot, but that has clearly been addressed. Ryan is a dinker and dunker and will never be " elite" as long as he is, and I don't care who makes a list.

Put it this way, I will take guys like Kurt Warner and Roger Staubachs words about Tony over Trent Dilfers words about Ryan. Cuz I am crazy like that.

[Joremarid]

lol @ people getting upset over this.

it means nothing.

[koolkat]

Romo has a TD/int ratio of 2-1, just like Ryan, and in 15 more career starts has 50 more TD's and 6,600 more yards.It's a given when you throw the ball deep more often, you will get more TD's AND int's. But its still 2-1 ratio and throwing the ball deeper is obviously harder than screen passes and 10 yard routes.

True Romo used to fumble a lot, but that has clearly been addressed. Ryan is a dinker and dunker and will never be " elite" as long as he is, and I don't care who makes a list.

Put it this way, I will take guys like Kurt Warner and Roger Staubachs words about Tony over Trent Dilfers words about Ryan. Cuz I am crazy like that.

PP if Ryan played for the Phins or the Raiders they would be talking about him like they did Romo on this MB. He's good but he hasn't done anything in the playoffs.

[pencilpusher. just because]

Never mind. I see they have Colt McCoy with his 2-6 record and his 6 TD's and 9 int's as an " average" QB, ahead of guys like Cutler, Hasselbeck and Bradford and McNabb. That's all I mneed to see to know the main thing this was built on was how much crack the guys who compiled this were doing.ven as bad a s some of these vets stunk it up last year, they were not Colt McCoy bad.

[Serge]

Romo has a TD/int ratio of 2-1, just like Ryan, and in 15 more career starts has 50 more TD's and 6,600 more yards.It's a given when you throw the ball deep more often, you will get more TD's AND int's. But its still 2-1 ratio and throwing the ball deeper is obviously harder than screen passes and 10 yard routes.

True Romo used to fumble a lot, but that has clearly been addressed. Ryan is a dinker and dunker and will never be " elite" as long as he is, and I don't care who makes a list.

Put it this way, I will take guys like Kurt Warner and Roger Staubachs words about Tony over Trent Dilfers words about Ryan. Cuz I am crazy like that.

I'm not saying I know how this system works, but I am saying that they didn't make a system specifically to make Matt Ryan look better, I'm guessing the system places a lot of emphasis on lost fumbles, sacks, and third down percentage and Ryan probably handled those three better than Romo since 2008. The fact that people are reacting to this like it's a personal afront to suggest that Ryan's regular-season performance grades out to an elite level in this system is absolutely baffling to me.

Besides, Ryan hardly throws any screen passes compared to Philly or New England, and it was only last year that he stayed under control and stopped throwing it down the field in poor situations like he did in his second year. The myth that Ryan is and always will be a dink-and-dunk passer was propagated on message boards rather than real assessments of his abilities.

[pencilpusher. just because]

PP if Ryan played for the Phins or the Raiders they would be talking about him like they did Romo on this MB. He's good but he hasn't done anything in the playoffs.

Oh I realize that. Believe me.They used to say Romo was only good because he had TO, then he has his over all best season when TO is dumped, and TO went on to suck in Buffalo. They would laugh at Ryan if he played fr another team.

[pencilpusher. just because]

I'm not saying I know how this system works, but I am saying that they didn't make a system specifically to make Matt Ryan look better, I'm guessing the system places a lot of emphasis on lost fumbles, sacks, and third down percentage and Ryan probably handled those three better than Romo since 2008. The fact that people are reacting to this like it's a personal afront to suggest that Ryan's regular-season performance grades out to an elite level in this system is absolutely baffling to me.

Besides, Ryan hardly throws any screen passes compared to Philly or New England, and it was only last year that he stayed under control and stopped throwing it down the field in poor situations like he did in his second year. The myth that Ryan is and always will be a dink-and-dunk passer was propagated on message boards rather than real assessments of his abilities.

I think that belief was propogated on the field last year when the longest pass was 46 yards. last in the league.