Welcome to the 17th annual NFL Prediction Tracker Awards. The purpose of these awards is
to help raise the publicity for those systems that have been superior in various qualities of
interest. The awards will be based entirely on the numbers I've monitored with my prediction
tracker web pages. In 2016 I tracked 72 rating/pediction systems that had data in he NFL for
at least half of the season.


BEST STRAIGHT UP WINNERS (Entire Season)

Winner: Kenneth Massey


straight up plot

 This season Kenneth Massey had the best record predicting games straight up, 178-87, 67.17%.
For the first time we had a game end in a tie with multiple systems predicting the tie outcome.
For those that predicted the tie I gave them a win.  That one win was what gave Massey
the season win over Pigskin Index.  67% doesn't seem that high but that is about typical
of what we see in a full season winner.  The 3 versions on the line finished 13-15th.
The season started off relatively week but from a month in to the end of the season the
records were trending up.


Winner 2016: Kenneth Massey
Winner 2015: Sagarin Golden Mean, YourLinx
Winner 2014: Dokter Entropy
Winner 2013: Ed Bemiss
Winner 2012: Sagarin Pure Points (predictive)
Winner 2011: Nationalsportsranking.com
Winner 2010: LVSC opening line
Winner 2009: JFM Power Ratings, Sportrends
Winner 2008: Ken Ashby
Winner 2007: Sonny Moore
Winner 2006: Least Absolute Value Regression
Winner 2005: Vegas Line (record 72.28%)
Winner 2004: Least Absolute Value Regression
Winner 2003: Beck Elo
Winner 2002: Herman Matthews
Winner 2001: JFM Power Ratings, ARGH Power Ratings
Winner 2000: CPA Rankings
Winner 1999: Kenneth Massey Ratings



  BEST AGAINST THE SPREAD (Entire Season)

Winner: Updated Line
Winner: Daniel Curry Index


ATS plot

 Some consider the true worth of a ranking system to be its record against the Vegas line.
Although that record for every game is probably not the best measurement but that is what I have.
2017 was a very bad year for playing systems against the spread.  Only 5 out of 71 finished
better than 50%, and two of those were versions of the line.
The updated line finished with the best record overall ATS at 55.621%  This is 3 seasons in a
row that the updated line has finished at least 55% or better.  The problem with this is how
do you use the current line to bet against a line from the past. Perhaps look for differences
between books to see which are moving?  What this is saying is that line movement closer to
game time is better than midweek. The other problem with the line is it doesn't suggest picks for
every game. The best with picks on all games was Daniel Curry Index, 139-123. 53.053%
The median performance for the year was way down around 46%.  My version of Elo finished at 39%, one
of the lowest records ever.  So overall 2017 goes down as the worst year ever for ATS records.


Winner 2016: Updated Line, Daniel Curry Index
Winner 2015: Logistic Regression (record 59.608%)
Winner 2014: Computer adjusted line, Turnover adjusted least squares regresssion
Winner 2013: Lou St. John  (record 59.449%)
Winner 2012: Least squares regression
Winner 2011: Pi-Rate Ratings
Winner 2010: Sagarin Predictor
Winner 2009: CPA Rankings.
Winner 2008: Nutshell Retro, Entropy System
Winner 2007: Sagarin Predictive
Winner 2006: Brian Gabrielle (record against updated line 59.0%)
Winner 2005: Least squares with team HFA
Winner 2004: Least Absolute Value Regression
Winner 2003: CPA Retro Rankings
Winner 2002: FreeSportsPlay.com
Winner 2001: CPA Rankings
Winner 2000: Yourlinx
Winner 1999: PerformanZ Ratings



  SMALLEST MEAN ABSOLUTE ERROR (Entire Season) 

Winner: Updated Line

ATS plot


  Deviation from the game score is the absoulte difference between the game prediction
and the actual result.  A value of zero would mean the score difference is predicted
exactly.  One property of a good system would be to minimize the system's average
game deviation.
  Right above I mentioned ATS had the worst season ever, Absolute error saw quite the opposite.
it was an extraordinary year. The Updated line wins for the secord year in a row and shatters
the old record by close to a full point.  This year was so good that the top 40 systems all did
better than the previous record low.


Winner 2016: Vegas Line (updated) (record 8.9607)
Winner 2015: Vegas Line (updated)
Winner 2014: Massey Ratings
Winner 2013: Ed Bemiss
Winner 2012: Vegas Line (updated)
Winner 2011: Dokter's Entropy System
Winner 2010: LVSC Opening line
Winner 2009: Vegas Line (updated)
Winner 2008: Ken Ashby
Winner 2007: Computer Adjusted Line
Winner 2006: Brian Gabrielle
Winner 2005: Vegas Line (updated)
Winner 2004: Vegas Line (updated)
Winner 2003: Vegas Line (updated)
Winner 2002: FreeSportsPlay.com
Winner 2001: Vegas Line (updated)
Winner 2000: Vegas Line (updated)
Winner 1999: Vegas Line (updated)



 SMALLEST BIAS (Entire Season)  

Winner: teamrankings.com

ATS plot

  Bias is a little different from deviation.  Deviation measures the distance
between a prediction and the actual result.  Bias combines distance and location
of the prediction.   Bias measures whether the predictions are too high or too low.
So if a system has an average bias of +0.25 that means that on average the system
gave 0.25 points too many to the home team.
  Teamrankings.com had the smallest bias over the full season, -0.00370.  The majority
of the systems underestimated the road team by an average of around half a point.


Winner 2016: TeamRanknings.com
Winner 2015: Lou St. John
Winner 2014: Sagarin Points Elo
Winner 2013: Game Time Decision
Winner 2012: PerformanZ Ratings
Winner 2011; Lee Burdorf
Winner 2010: Stephen Kerns
Winner 2009; System Median
Winner 2008: ARGH Power Ratings
Winner 2007: CPA Rankings
Winner 2006: Tom Benson
Winner 2005: Sagarin Elo (record 0.001)
Winner 2004: System Average
Winner 2003: CPA Rankings
Winner 2002: Sagarin Pure Points
Winner 2001: ARGH Power Ratings
Winner 2000: Pigskin Index
Winner 1999: Flyman Ratings



 MOST ACCURATE SYSTEM - SMALLEST MEAN SQUARE ERROR (Entire Season)


Winner: Midweek Line

ATS plot

 This award is based on mean square error. Mean square error takes
into account both deviation and bias and is a commonly used measure
of evaluating estimators.
  Like we saw with absolute error the mean square errors were off the charts good this
past season.  The midweek line led that way at 136.25300.  More than 20 points better
than the old record set by Lou St. John back in 2013.  This season 49 out of the 71
systems did better than the previous all time record.  My take on this as I watched
the season progress was that there seemed to be more parity, or more toss up games than
ever this year.  This lead to easier to predict smaller margins and harder to predict
against the spread.


Winner 2016: Midweek Line
Winner 2015: Dokter Entropy
Winner 2014: Computer Adjusted Line
Winner 2013: Lou St. John (record 157.3140)
Winner 2012: Vegas Line (updated)
Winner 2011: Dokter Entropy System
Winner 2010: Computer Adjusted Line
Winner 2009: Vegas Line (updated)
Winner 2008: Computer Adjusted Line
Winner 2007: Computer Adjusted Line
Winner 2006: Computer Adjusted Line
Winner 2005: Vegas Line (updated)
Winner 2004: Vegas Line (updated)
Winner 2003: Vegas Line (opening)
Winner 2002: FreeSportsPlay.com
Winner 2001: Vegas Line
Winner 2000: Pigskin Index




            SECOND HALF AWARDS 


Some systems donít begin publishing rankings until every team has played a couple of games, while
the season-long systems often rely on preset starting values in the early part of the season. For
those reasons, I like to look at the results over the second half of the season when these systems
have had time to become 'burned in' to  the season's data. Second half data consists of all games
from week 10 through the Super Bowl.


  BEST STRAIGHT UP WINNERS (Second Half)

Winner: Midweek Line
Winner: Donchess Inference
Winner: System Average
Winner: Massey Ratings
Winner: Jeff Bihl


straight up plot

  It was crowded at the top this year which is how this category played out the first ten years.
First place was shared by the midweek Line, Donchess Inference, the system average, Kenneth Massey,
and Jeff Bihl.  The first four were 97-37, 72.388%.  Bihl Actually finished higher but missed a week.
Not counting that week it would have been a five way tie. The midweek line finished 3 games better
than both the opening and closing line.

Winner 2016: Midweek Line, Donchess Inference, System Average, Massey Ratings, Bihl Rankings
Winner 2015: Talisman Red
Winner 2014: Logistic Regression
Winner 2013: Ed Bemiss
Winner 2012: Sonny Moore, Kasulis Enhanced Spread
Winner 2011: Turnover adjusted least squares
Winner 2010: LVSC opening line
Winner 2009: The Vegas line, Computer adjusted line, Ashby AccuRatings, StatFox, Upset Report, CPA Rankings
Winner 2008: Nutshell Girl
Winner 2007: Computer Adjusted Line, Vegas Line (updated)
Winner 2006: Least Absolute Value Regression
Winner 2005: Vegas Line (updated), Nutshell Sports (record 100-37, 72.99%)
Winner 2004: Pigskin Index
Winner 2003: JFM Power Ratings, Kasulis Enhanced Spread, Beck Elo
Winner 2002: Least Absolute Value Regression, JFM Power Ratings, CPA Rankings
Winner 2001: Pythagorean Ratings, Elo Ratings, Ken Massey Ratings
Winner 2000: PerformanZ Ratings, Scoring Effeciency
Winner 1999: Ken Massey, Ed Kambor, Pythagorean Ratings



  BEST AGAINST THE SPREAD (Second Half)

Winner:  Updated line
Winner: ProComputerGambler

straight up plot

  As with the full season, in the second half the updated line had the best ATS performance
and the results as a group were very poor. The line had a 54% record.  ProComputerGambler.com
was the best with picks across all games, 70-61, 53.435%  Only 6 out of 69 systems finished
better than 50% ATS. Last year about the same number finished better than 60%. For a comparison,
last season the computer adjusted line was 65%.  Despite their name, covers.com finished with a
historically low 36%. The system average was only 40%. Usually trustworthy Sagarin was also only 40%.

Winner 2016: Updated Line, ProComputerGambler
Winner 2015: Computer adjusted line, Dokter Entropy.
Winner 2014: Least Squares regression, computer adjusted line (record  65.476% in 63% of games)
Winner 2013: Sonshine forecast (record 68-39, 63.551%)
Winner 2012: Sagarin Predictor, Sagarin Overall
Winner 2011: Kambour Ratings
Winner 2010: Sagarin Predictor, Edward Kambour
Winner 2009: Grid Iron Gold
Winner 2008: Ken Ashby
Winner 2007: Least Squares with Team Specific HFA
Winner 2006: CPA Rankings
Winner 2005: Tom Benson
Winner 2004: ARGH Power Ratings
Winner 2003: Logistic Regression, Viacheslav Ugolnikov
Winner 2002: Least Absolute Value Regression (record against updated line 78-52, 60.0%)
Winner 2001: Stat Fox
Winner 2000: Pythagorean Ratings
Winner 1999: Least squares using team specific home field advantages


  SMALLEST MEAN ABSOLUTE ERROR (Second Half)

Winner: Updated Line

straight up plot

The updated line had the lowest mean error in the seond half, 9.1903. This is an new all
time record.  Over half a point better than the previous record set by IronRank back in
2013.  This number is a point and a half better than the winner from last season.  The
errors were small for both the full and half seasons.  29 out of 69 systems bettered the
previous record.

Winner 2016: Vegas Line (updated)  (Recod 9.1903)
Winner 2015: Talisman Red
Winner 2014: Least Squares regression
Winner 2013: Ironrank.com
Winner 2012: Lou St. John
Winner 2011: Turnover adjusted least squares
Winner 2010: Massey Ratings
Winner 2009: Vegas Line (updated)
Winner 2008: The Entropy System
Winner 2007: Vegas Line (updated)
Winner 2006: Jeff Self
Winner 2005: Vegas Line (updated)
Winner 2004: Least Squares Regression
Winner 2003: Vegas Line (updated)
Winner 2002: FreeSportsPlays.com
Winner 2001: Vegas Line (updated)
Winner 2000: Pythagorean Ratings


  SMALLEST BIAS (Second Half)    

Winner: ESPN FPI

straight up plot

  ESPN FPI had the smallest bias in the second half at 0.18680.  Jeff Bihl had a lower value
but if you removed the week he missed from everyone else he would not have finished on top.
It is suprising that the error margins were record setting but the biases were quite bad.
For the second half they were overwhelmingly negative, with only 3 systems giving to many
points to the home team. Last season these numbers where overwhelmingly positive.

Winner 2016: ESPN FPI
Winner 2015: LS - w/ team HFA
Winner 2014: Locksmith picks
Winner 2013: Covers.com
Winner 2012: Nutshell Sports
Winner 2011: Lee Burdorf
Winner 2010: CPA Retro Rankings
Winner 2009; Game Time Decision
Winner 2008: CPA Rankings
Winner 2007: Pigskin Index (record 0.0077)
Winner 2006: CPA Retro Rankings
Winner 2005: Kasulis Enhanced Spread
Winner 2004: Tom Benson Gridmarks
Winner 2003: CPA Rankings
Winner 2002: StatFox.com
Winner 2001: Monte Carlo Markov Chain
Winner 2000: Flyman Performance Ratings
Winner 1999: Flyman Performance Ratings




  MOST ACCURATE SYSTEM - SMALLEST MEAN SQUARE ERROR (Second Half)


Winner: Computer Adjusted Line

straight up plot

 The computer adjusted line had the smallest mean square error in the second half and also sets
a new record with 147.025.  This is 11 points better than the old record set by Steven Jens in
2013.  30 of the 69 systems bettered the previous record.

Winner 2016: Computer Adjusted Line (record 147.025)
Winner 2015: Talisman Red
Winner 2014: Ironrank.com
Winner 2013: Steven Jens
Winner 2012: Vegas Line (updated)
Winner 2011: Turnover adjusted least squares
Winner 2010: Jeff Sagarin
Winner 2009: Vegas Line (updated)
Winner 2008: Hank Trexler
Winner 2007: Vegas Line (updated)
Winner 2006: CPA Rankings
Winner 2005: Vegas Line (updated)
Winner 2004: Computer Adjusted Line
Winner 2003: Viacheslav Ugolnikov
Winner 2002: FreeSportsPlay.com
Winner 2001: Hank's Power Ratings
Winner 2000: Pythagorean Ratings



   BEST PREDICTIVE SYSTEM (Entire Season)

Winner: Game-Time Decision

To come up with the best overall predictive system I give each system
points for how well they do in all of the above categories.  I then sum up
the points and the system with the highest total is dubbed predictive system
of the year.  So this award goes to the system that is the most well rounded.
One flaw can totally take a system out of the running.
  For those that had pedictions for the full season, or at least most of the
season, the best overall predictive system was Game Time Decision.  GTD was
top 10 in every category exect for ATS where he was 12th.  Nobody ran away with
it this year.  Othere than the line the the closest competitor was Pigskin Index.



Winner 2016: Game Time Decision
Winner 2015: Doktor Entropy
Winner 2014: Pythagorean Ratings
Winner 2013: Ed Bemiss
Winner 2012: Sagarin Pure Points
Winner 2011: nationalsportsrankings.com
Winner 2010: LVSC opening line
Winner 2009: Vegas Line
Winner 2008: Ken Ashby
Winner 2007: Jeff Sagarin
Winner 2006: Brian Gabrielle
Winner 2005: Vegas Line
Winner 2004: Least Absolute Value Regression
Winner 2003: Beck Elo
Winner 2002: Matthews Grid
Winner 2001: CPA Rankings
Winner 2000: Ed Kambour


  BEST PREDICTIVE SYSTEM (Second Half)



Winner: ESPN FPI

  The top point getter inthe second half was ESPN FPI.  ESPN struggled straight up
but was top 10 in all the other categories. Technically the updated line would have
won both the full season and second half but I tend to give this to ratings when they
are reasonably close.  Second place was Dwiggins dpdsdogs.


Winner 2016: ESPN FPI
Winner 2015: Talisman Red
Winner 2014: GameTimeDecison
Winner 2013: IronRank.com
Winner 2012: Sagarin Pure Points
Winner 2011: Edward Kambour Ratings
Winner 2010: LVSC opening line
Winner 2009: Game Time Decision
Winner 2008: Dokter Entropy System
Winner 2007: Cover 81
Winner 2006: CPA Retro
Winner 2005: Hank Trexler
Winner 2004: ARGH Power Ratings
Winner 2003: CPA Rankings
Winner 2002: Kenneth Massey
Winner 2001: Monte Carlo Markov Chain
Winner 2000: Pythagorean Ratings