Welcome to the 12th annual NFL Prediction Tracker Awards. The purpose of these awards is
to help raise the publicity for those systems that have been superior in various qualities of
interest. Most awards will be based entirely on the numbers I've monitored with my prediction
tracker web pages. In 2011 the number of NFL systems went up by 8 to a total of 73 systems.



BEST STRAIGHT UP WINNERS (Entire Season)


Winner: nationalsportsrankings.com


  This season we had one system, Nutshell Sports, have the most games correct but a
different system, nationalsportsrankings.com, have the best percentage.
NSR was 175-76, 69.721%.  Nutshell was 184-83, 68.914%.  NSR was missing week one.
After throwing out week 1 NSR was two games better than Nutshell.  So I'm giving
this the nationalsportsranking.com and will give Nutshell and honorable mention.
  What is odd about this year is that you have to drop all the way down to number 29
before you find the Vegas line which was 65.5% this year.  Maybe that sounds low but
it is actually an improvement over what it did last year which was 64.4% and good enough
for second place.


Winner 2011: Nationalsportsranking.com
Winner 2010: LVSC opening line
Winner 2009: JFM Power Ratings, Sportrends
Winner 2008: Ken Ashby
Winner 2007: Sonny Moore
Winner 2006: Least Absolute Value Regression
Winner 2005: Vegas Line (record 72.28%)
Winner 2004: Least Absolute Value Regression
Winner 2003: Beck Elo
Winner 2002: Herman Matthews
Winner 2001: JFM Power Ratings, ARGH Power Ratings
Winner 2000: CPA Rankings
Winner 1999: Kenneth Massey Ratings




  BEST AGAINST THE SPREAD (Entire Season)

Winner:  Pi-Rate Ratings

 Some consider the true measure of a ranking system to be its record against the Vegas line.
Although that record for every single game is probably not the best measurement but that is
what I have.
  It was a very good year for the computer ratings picking against the spread.  I suspect
that the line's poor performance relative to others straight up was a major factor.
The Pi-Rate Ratings led the way with a record of 143-110, 56.522%.  This is about where the
winner usually is but the difference was that 6 different systems were better than 55% and
18 were better than 53%.
  I tried something new this season by posting predicted probabilities of covering the spread
(which were based on the computer average) on my blog page.  I kept track of how well the top 3
picks did each week.  Over the course of the season the top 3 games of the week went 33-18,
or 64.7%. So there were definitely opportunities to use the computers to beat the spread this
season.


Winner 2011: Pi-Rate Ratings
Winner 2010: Sagarin Predictor
Winner 2009: CPA Rankings.
Winner 2008: Nutshell Retro, Entropy System
Winner 2007: Sagarin Predictive
Winner 2006: Brian Gabrielle (record 59.0%)
Winner 2005: Least squares with team HFA
Winner 2004: Least Absolute Value Regression
Winner 2003: CPA Retro Rankings
Winner 2002: FreeSportsPlay.com
Winner 2001: CPA Rankings
Winner 2000: Yourlinx
Winner 1999: PerformanZ Ratings





  SMALLEST MEAN ABSOLUTE ERROR (Entire Season) 

Winner: The Entropy System 

  Deviation from the game score is the difference between the game prediction
and the actual result.  A value of zero would mean the score difference is predicted
exactly.  One property of a good system would be to minimize the system's average
game deviation.
  Nationalsportsrankings.com was also the apparent leader in this category, however,
for this one, once taking out week one he no longer remained on top. So the lead goes
to the best over all weeks which was Jon Dokter's Entropy System, which had a mean
absolute error of 10.5258.   This just barely edges out Wonderdog Sports' value of
10.5281.   The line finished in 5th place.



Winner 2011: Dokter's Entropy System
Winner 2010: LVSC Opening line
Winner 2009: Vegas Line (updated)
Winner 2008: Ken Ashby
Winner 2007: Computer Adjusted Line
Winner 2006: Brian Gabrielle
Winner 2005: Vegas Line (updated) (record 9.9513)
Winner 2004: Vegas Line (updated)
Winner 2003: Vegas Line (updated)
Winner 2002: FreeSportsPlay.com
Winner 2001: Vegas Line (updated)
Winner 2000: Vegas Line (updated)
Winner 1999: Vegas Line (updated)




 SMALLEST BIAS (Entire Season)  


Winner: Lee Burdorf

  Bias is a little different from deviation.  Deviation measures the
distance between a prediction and the actual result.  Bias combines
distance and location of the prediction.   Bias measures whether the
predictions are too high or too low.  So if a system has an
average bias of +0.25 that means that on average the system
gave 0.25 points too many to the home team.  But I want to add that
some people are making this out to be nothing  more than a measure of
how good a system measures home field advantage.  But that is not the
whole story. I could just as easily make the reference the favorite.
The bottom line is the closer the bias is to zero the more accurate the
system is on average, regardless of home field advantage.
  The biases as a whole were decidedly negative this year.  For the full
season only one system, Steven Jens, had a positive bias.  But the closest
to zero was by Lee Burdorf at -0.0391.


Winner 2011; Lee Burdorf
Winner 2010: Stephen Kerns
Winner 2009; System Median
Winner 2008: ARGH Power Ratings
Winner 2007: CPA Rankings
Winner 2006: Tom Benson
Winner 2005: Sagarin Elo (record 0.001)
Winner 2004: System Average
Winner 2003: CPA Rankings
Winner 2002: Sagarin Pure Points
Winner 2001: ARGH Power Ratings
Winner 2000: Pigskin Index
Winner 1999: Flyman Ratings




 MOST ACCURATE SYSTEM - SMALLEST MEAN SQUARE ERROR (Entire Season)


Winner: The Entropy System 

  This award is based on mean square error. Mean square error takes
into account both deviation and bias and is perhaps the most commonly
used measure of evaluating estimators.
  This is actually kind of significant.  Either the line, or the computer adjusted
line (which is based on the line) win this award 9 of the last 10 years.  This year
those two finish 3rd and forth place.  Nationalsportsrankings.com and Dokter's
Entropy System finish better. And once again removing week one NSR drops down. So
the most accurate system of 2011 goes to TimeTravelSports.com's Entropy System.
Entropy had a MSE of 187.16500.


Winner 2011: Dokter Entropy System
Winner 2010: Computer Adjusted Line
Winner 2009: Vegas Line (updated)
Winner 2008: Computer Adjusted Line
Winner 2007: Computer Adjusted Line
Winner 2006: Computer Adjusted Line
Winner 2005: Vegas Line (updated)
Winner 2004: Vegas Line (updated) (record 166.808)
Winner 2003: Vegas Line (opening)
Winner 2002: FreeSportsPlay.com
Winner 2001: Vegas Line
Winner 2000: Pigskin Index




            SECOND HALF AWARDS 


Some systems donít begin publishing rankings until every team has played a couple of games, while
the season-long systems often rely on preset starting values in the early part of the season. For
those reasons, I like to look at the results over the second half of the season when these systems
have had time to become 'burned in' to  the season's data. Second half data consists of all games
from week 10 through the Super Bowl.



  BEST STRAIGHT UP WINNERS (Second Half)


Winner: Turnover adjusted least squares regression

  The system that was the best straight up over the second half was my new turnover adjusted
least squares regression.  This system was 96-41, 70.073% during the second half.  The Dunkel
Index and Ridge regression were 1 game back.  The Vegas line was way down at #40.
  During the offseason last year I was thinking about some discussions I had had with
Jim Ashburn of Atomic Football.  Basically my belief has been that the Vegas line is unbiased
and therefore which team covers the spread was a random event.  So it wouldn't matter if a
computer system is 1 point greater than the line or 7 points, each would be equally likely
(roughly 50%) to be correct against the spread.  So I thought about how to verify by
beliefs.  Pondering that lead me to look at the line.  The line is approximately unbiased,
if you look at the bias numbers they are historically centered around zero. That in itself
goes a long way towards explaining why it is hard to be much better than 50%.  But then if
you look at the absolute error there appears to be a disconnect. On average the bias is close
to zero while the average absolute error is 10-11 points.  So even though the difference between
the line and actual score is close to zero on average, the line is off by 10 points or more
more than 50% of the time.  That is mind blowing when you think about it.  Shouldn't that
imply that there is a lot of opportunities to beat the spread? If the line is off by so much
so often why can't someone or some system consistently find those holes?  That lead me to
think about how can you find those holes or explain that somewhat extreme variability. If the
difference between the line and the actual score is truly zero with some error, is there anything
that can explain that error or is it just random noise.  It seemed clear that the most likely
candidate was turnovers, which are generally considered to be random bounces of the ball.
Random error, random turnovers, sounds like there could be a connection.  The problem was
I don't collect individual game statistics so I couldn't investigate this idea.  I eventually
found a source of data so that I could look at the 2010 season.  What I found was that
the turnover margin in a game explained roughly 40% of the difference between the line and
the actual outcome.  I got very excited.  The 2011 preseason was about to start so I needed
come up with a system that incorporated the turnovers along with the scores.  So what I did
was very simple.  I just added it as a new variable in the least squares regression model
that I have been running for years. For these models the outcome is the score differential.
The variables are a matrix of the games.  For each game a variable for the home team is equal
to one, and the variable for the road team is equal to negative one.  All other team variables
have a value of zero.  To this I added the turnover margin for the game.  I knew the results early
in season would be meaningless because this system is based only on games of the current year.
So it could take some time before it became stable.  When it did kick in it really kicked in.
As you will see, it dominated the second half accuracy categories.



Winner 2011: Turnover adjusted least squares
Winner 2010: LVSC opening line
Winner 2009: The Vegas line, Computer adjusted line, Ashby AccuRatings, StatFox, Upset Report, CPA Rankings
Winner 2008: Nutshell Girl
Winner 2007: Computer Adjusted Line, Vegas Line (updated)
Winner 2006: Least Absolute Value Regression
Winner 2005: Vegas Line (updated), Nutshell Sports (record 100-37, 72.99%)
Winner 2004: Pigskin Index
Winner 2003: JFM Power Ratings, Kasulis Enhanced Spread, Beck Elo
Winner 2002: Least Absolute Value Regression, JFM Power Ratings, CPA Rankings
Winner 2001: Pythagorean Ratings, Elo Ratings, Ken Massey Ratings
Winner 2000: PerformanZ Ratings, Scoring Effeciency
Winner 1999: Ken Massey, Ed Kambor, Pythagorean Ratings




  BEST AGAINST THE SPREAD (Second Half)

Winner:  Kambour Ratings

  Based on some of what I said above, I believe one of the reasons why this was
a good year against the spread was the fact that the bias of the line was larger
than normal. It was about -1 for the season as a whole and even larger in the second half.
We didn't see record setting numbers but we had an unusually large number of systems
getting 53, 54 or more percent.  Dr. Edward Kambour's Ratings were the best in the second
half with a record of 74-57, 56.489%.   For the overall season there weren't any really
bad systems against the spread, so interestingly there were just as many bad ones during
the second half as there were good ones.  For example Nutshell Combo and Game Time Decision
was only about 42%. If you picked opposite from those you would have done pretty well.



Winner 2011: Kambour Ratings
Winner 2010: Sagarin Predictor, Edward Kambour
Winner 2009: Grid Iron Gold
Winner 2008: Ken Ashby
Winner 2007: Least Squares with Team Specific HFA
Winner 2006: CPA Rankings
Winner 2005: Tom Benson
Winner 2004: ARGH Power Ratings
Winner 2003: Logistic Regression, Viacheslav Ugolnikov
Winner 2002: Least Absolute Value Regression (record 78-52, 60.0%)
Winner 2001: Stat Fox
Winner 2000: Pythagorean Ratings
Winner 1999: Least squares using team specific home field advantages





  SMALLEST MEAN ABSOLUTE ERROR (Second Half)


Winner: Turnover adjusted least squares regression

  The line had won this award every other year going back to when I first started but
that pattern comes to an end this year. The line only managed 5th place.  My new
turnover adjusted least squares led the second half with a mean error of 10.6231.
As mentioned above, I added turnover margin to the model in an attempt to adjust
for some of the random error.  I guess it was too much to hope for that explaining
40% of the error on past games would lead to a 40% reduction on future games.  But
it did take first place by a decent margin.  It will be interesting to see if this
was just luck or it will continue to put up good numbers.  Wonderdog Sports actually
had an extremely good season as well.


Winner 2011: Turnover adjusted least squares
Winner 2010: Massey Ratings
Winner 2009: Vegas Line (updated)
Winner 2008: The Entropy System
Winner 2007: Vegas Line (updated)
Winner 2006: Jeff Self
Winner 2005: Vegas Line (updated) (record 9.9489)
Winner 2004: Least Squares Regression
Winner 2003: Vegas Line (updated)
Winner 2002: FreeSportsPlays.com
Winner 2001: Vegas Line (updated)
Winner 2000: Pythagorean Ratings




  SMALLEST BIAS (Second Half)    


Winner: Lee Burdorf

  None of the systems that participated over the entire second half had a positive
bias.  This is opposite of last year when all but a handful were positive.
This trend of the systems being almost all positive or all negative has flip flopped
from year to year for 5 seasons now.  The lowest was Lee Burdorf, -0.10070.  Burdorf
also had the smallest bias over the full season.  I'm not sure what that means.
Perhaps the odds makers are continuously over compensating for the prior year, or
perhaps it has to do with the scheduling, as matchups generally flip home/away
locations each year.

Winner 2011: Lee Burdorf
Winner 2010: CPA Retro Rankings
Winner 2009; Game Time Decision
Winner 2008: CPA Rankings
Winner 2007: Pigskin Index (record 0.0077)
Winner 2006: CPA Retro Rankings
Winner 2005: Kasulis Enhanced Spread
Winner 2004: Tom Benson Gridmarks
Winner 2003: CPA Rankings
Winner 2002: StatFox.com
Winner 2001: Monte Carlo Markov Chain
Winner 2000: Flyman Performance Ratings
Winner 1999: Flyman Performance Ratings




  MOST ACCURATE SYSTEM - SMALLEST MEAN SQUARE ERROR (Second Half)


Winner: Turnover adjusted least squares regression


  The most accurate system for the second half, defined by having the smallest mean square error
was turnover adjusted least squares with an MSE of  180.68300. Pi-Rate Ratings came in second place.
After several seasons of the mean square errors getting bigger and bigger we have now seen a
decrease for two years in a row.


Winner 2011: Turnover adjusted least squares
Winner 2010: Jeff Sagarin
Winner 2009: Vegas Line (updated)
Winner 2008: Hank Trexler
Winner 2007: Vegas Line (updated)
Winner 2006: CPA Rankings
Winner 2005: Vegas Line (updated)
Winner 2004: Computer Adjusted Line
Winner 2003: Viacheslav Ugolnikov
Winner 2002: FreeSportsPlay.com
Winner 2001: Hank's Power Ratings
Winner 2000: Pythagorean Ratings (record 165.954)





   BEST PREDICTIVE SYSTEM (Entire Season)


Winner: nationalsportsrankings.com

To come up with the best overall predictive system I give each system
points for how well they do in all of the above categories.  I then sum up
the points and the system with the highest total is dubbed predictive system
of the year.  So this award goes to the system that is the most well rounded
One flaw can totally take a system out of the running.
  The best predictive system for the full season was nationaalsportsranks.com.
Just based on the number of separate categories he was a leading contender in this
is not much of a surprise.  If there is a surprise it is that StatFox was relatively
close in second place.   Other than the line a couple of time we have never had
a system win best predictive system multiple times.


Winner 2011: nationalsportsrankings.com
Winner 2010: LVSC opening line
Winner 2009: Vegas Line
Winner 2008: Ken Ashby
Winner 2007: Jeff Sagarin
Winner 2006: Brian Gabrielle
Winner 2005: Vegas Line
Winner 2004: Least Absolute Value Regression
Winner 2003: Beck Elo
Winner 2002: Matthews Grid
Winner 2001: CPA Rankings
Winner 2000: Ed Kambour



  BEST PREDICTIVE SYSTEM (Second Half)


Winner:  Kambour Ratings

  Now this one does come as a surprise.  Turnover adjusted least squares won three second
half awards so I thought it was safe to assume it was going to be an easy winner for
best predictive system over the second half of the season.  I guess that is why they
play the game as they say.  Edward Kambour's ratings comes out on top.  The turnover
adjusted least squares system came in second.  If you look at the two, Kambour was
above average in every category.  The turnover adjusted model was exception in 4 of
the five categories but in that fifth one, it was below average.  A similar thing happened
last season to Game Time Decision.  The whole point of being the best overall system
is to be good at everything not just some things.   There have been 12 different winners
over 12 seasons.  I'm not really sure what that says other than no one system is consistently
the best.


Winner 2011: Edward Kambour Ratings
Winner 2010: LVSC opening line
Winner 2009: Game Time Decision
Winner 2008: Dokter Entropy System
Winner 2007: Cover 81
Winner 2006: CPA Retro
Winner 2005: Hank Trexler
Winner 2004: ARGH Power Ratings
Winner 2003: CPA Rankings
Winner 2002: Kenneth Massey
Winner 2001: Monte Carlo Markov Chain
Winner 2000: Pythagorean Ratings




 RETRO AWARDS 

   I am no longer going to give retro awards