Welcome to the 18th annual NFL Prediction Tracker Awards. The purpose of these awards is
to help raise the publicity for those systems that have been superior in various qualities of
interest. The awards will be based entirely on the numbers I've monitored with my prediction
tracker web pages. In 2017 I tracked 65 rating/pediction systems that had data in he NFL for
at least half of the season.


BEST STRAIGHT UP WINNERS (Entire Season)

Winner: dratings.com


straight up plot

 This season dratings.com, aka Donchess Inference,  had the best record predicting games straight
up, 183-84-87, 68.539%.  Lou St. John was one game back and the updated line and teamrankings.com
were two games back.  The overall trend this season looks much like last year.  The numbers are
weak for the first 4 to 6 weeks and then begin to improve through the end of the season.


Winner 2017: Dratings.com, 68.5%
Winner 2016: Kenneth Massey, 67.2%
Winner 2015: Sagarin Golden Mean, YourLinx
Winner 2014: Dokter Entropy
Winner 2013: Ed Bemiss
Winner 2012: Sagarin Pure Points (predictive)
Winner 2011: Nationalsportsranking.com
Winner 2010: LVSC opening line
Winner 2009: JFM Power Ratings, Sportrends
Winner 2008: Ken Ashby
Winner 2007: Sonny Moore
Winner 2006: Least Absolute Value Regression
Winner 2005: Vegas Line (record 72.28%)
Winner 2004: Least Absolute Value Regression
Winner 2003: Beck Elo
Winner 2002: Herman Matthews
Winner 2001: JFM Power Ratings, ARGH Power Ratings
Winner 2000: CPA Rankings
Winner 1999: Kenneth Massey Ratings



  BEST AGAINST THE SPREAD (Entire Season)

Winner: Updated Line
Winner: PerformanZ Ratings

ATS plot

 Some consider the true worth of a ranking system to be its record against the Vegas line.
Although that record for every game is probably not the best measurement but that is what I have.
2018 was a pretty good year for playing systems against the spread.  50 systems did better than
50%.  Compare that to only 5 out of 71 last season.
The updated line finished with the best record overall ATS at 60%, 99-66.  This is 4 seasons in a
row that the updated line has finished at least 55% or better and second consecutive season
to technically be the best percentage.  The problem with this is how do you use the current
line to bet against a line from the past.  What this is saying is that line movement closer to
game time is better than midweek. The other problem with the line is it doesn't suggest picks for
every game. The best with picks on all games was my own PerformanZ Ratings, 149-110. 57.529%
No systems were overly bad.  The worst was only 47%.


Winner 2017: Updated Line 60%, PerformanZ Ratings 57.529%
Winner 2016: Updated Line 55.621%, Daniel Curry Index 53.053%
Winner 2015: Logistic Regression (record 59.608%)
Winner 2014: Computer adjusted line, Turnover adjusted least squares regresssion
Winner 2013: Lou St. John  59.449%
Winner 2012: Least squares regression
Winner 2011: Pi-Rate Ratings
Winner 2010: Sagarin Predictor
Winner 2009: CPA Rankings.
Winner 2008: Nutshell Retro, Entropy System
Winner 2007: Sagarin Predictive
Winner 2006: Brian Gabrielle (record against updated line 59.0%)
Winner 2005: Least squares with team HFA
Winner 2004: Least Absolute Value Regression
Winner 2003: CPA Retro Rankings
Winner 2002: FreeSportsPlay.com
Winner 2001: CPA Rankings
Winner 2000: Yourlinx
Winner 1999: PerformanZ Ratings


  SMALLEST MEAN ABSOLUTE ERROR (Entire Season) 

Winner: Updated Line

ATS plot


  Deviation from the game score is the absoulte difference between the game prediction
and the actual result.  A value of zero would mean the score difference is predicted
exactly.  One property of a good system would be to minimize the system's average
game deviation.
  The updated line had the smallest absolute error for the third year in a row, 10.1124.
This is a full point below last year's record setting performance.  The top 4 were all
versions of the line.  The best non line system was Donchess Inference at 10.3114.


Winner 2017: Vegas Line (updated) 10.1124
Winner 2016: Vegas Line (updated) (record 8.9607)
Winner 2015: Vegas Line (updated)
Winner 2014: Massey Ratings
Winner 2013: Ed Bemiss
Winner 2012: Vegas Line (updated)
Winner 2011: Dokter's Entropy System
Winner 2010: LVSC Opening line
Winner 2009: Vegas Line (updated)
Winner 2008: Ken Ashby
Winner 2007: Computer Adjusted Line
Winner 2006: Brian Gabrielle
Winner 2005: Vegas Line (updated)
Winner 2004: Vegas Line (updated)
Winner 2003: Vegas Line (updated)
Winner 2002: FreeSportsPlay.com
Winner 2001: Vegas Line (updated)
Winner 2000: Vegas Line (updated)
Winner 1999: Vegas Line (updated)




 SMALLEST BIAS (Entire Season)  

Winner: Median of all system preditions

ATS plot

  Bias is a little different from deviation.  Deviation measures the distance
between a prediction and the actual result.  Bias combines distance and location
of the prediction.   Bias measures whether the predictions are too high or too low.
So if a system has an average bias of +0.25 that means that on average the system
gave 0.25 points too many to the home team.
  The median value of all system predictions  had the smallest bias over the full
season, -0.01190.  Sagarin points was next at -0.03290. This year the values were
fairly well balanced around zero rather than all being on one side like we often see.


Winner 2017: System Median,  -0.01190
Winner 2016: TeamRanknings.com,  -0.00370
Winner 2015: Lou St. John
Winner 2014: Sagarin Points Elo
Winner 2013: Game Time Decision
Winner 2012: PerformanZ Ratings
Winner 2011; Lee Burdorf
Winner 2010: Stephen Kerns
Winner 2009; System Median
Winner 2008: ARGH Power Ratings
Winner 2007: CPA Rankings
Winner 2006: Tom Benson
Winner 2005: Sagarin Elo (record 0.001)
Winner 2004: System Average
Winner 2003: CPA Rankings
Winner 2002: Sagarin Pure Points
Winner 2001: ARGH Power Ratings
Winner 2000: Pigskin Index
Winner 1999: Flyman Ratings



 MOST ACCURATE SYSTEM - SMALLEST MEAN SQUARE ERROR (Entire Season)


Winner: Updated Line

ATS plot

 This award is based on mean square error. Mean square error takes
into account both deviation and bias and is a commonly used measure
of evaluating estimators.
  Like we saw with absolute error the mean square errors were off the charts good this
past season.  The updated line led that way at  173.55700.  The best non line system
was Sports Cruncher at 181.12500.


Winner 2017: Updated Line, 173.55700
Winner 2016: Midweek Line  (record 136.25300)
Winner 2015: Dokter Entropy
Winner 2014: Computer Adjusted Line
Winner 2013: Lou St. John  157.3140
Winner 2012: Vegas Line (updated)
Winner 2011: Dokter Entropy System
Winner 2010: Computer Adjusted Line
Winner 2009: Vegas Line (updated)
Winner 2008: Computer Adjusted Line
Winner 2007: Computer Adjusted Line
Winner 2006: Computer Adjusted Line
Winner 2005: Vegas Line (updated)
Winner 2004: Vegas Line (updated)
Winner 2003: Vegas Line (opening)
Winner 2002: FreeSportsPlay.com
Winner 2001: Vegas Line
Winner 2000: Pigskin Index




            SECOND HALF AWARDS 


Some systems donít begin publishing rankings until every team has played a couple of games, while
the season-long systems often rely on preset starting values in the early part of the season. For
those reasons, I like to look at the results over the second half of the season when these systems
have had time to become 'burned in' to  the season's data. Second half data consists of all games
from week 10 through the Super Bowl.


  BEST STRAIGHT UP WINNERS (Second Half)


Winner: Lou St. John
Winner: Least Absolute Value Regression

  This came close to being very messy.  We ended up with two systems, Lou St. John and Least
absolute value regression tied for for first place with a record of 98-37, 72.593%.  But then
we had 11 other systems tied with a record of only one game behind. One of those 11 was the updated
line.  So the two winners are the only two system that outperformed the updated line in the second
half.


Winner 2017: Lou St. John, Least absolute value regression  72.593%
Winner 2016: Midweek Line, Donchess Inference, System Average, Massey Ratings, Bihl Rankings   72.388%
Winner 2015: Talisman Red
Winner 2014: Logistic Regression
Winner 2013: Ed Bemiss
Winner 2012: Sonny Moore, Kasulis Enhanced Spread
Winner 2011: Turnover adjusted least squares
Winner 2010: LVSC opening line
Winner 2009: The Vegas line, Computer adjusted line, Ashby AccuRatings, StatFox, Upset Report, CPA Rankings
Winner 2008: Nutshell Girl
Winner 2007: Computer Adjusted Line, Vegas Line (updated)
Winner 2006: Least Absolute Value Regression
Winner 2005: Vegas Line (updated), Nutshell Sports (record 100-37, 72.99%)
Winner 2004: Pigskin Index
Winner 2003: JFM Power Ratings, Kasulis Enhanced Spread, Beck Elo
Winner 2002: Least Absolute Value Regression, JFM Power Ratings, CPA Rankings
Winner 2001: Pythagorean Ratings, Elo Ratings, Ken Massey Ratings
Winner 2000: PerformanZ Ratings, Scoring Effeciency
Winner 1999: Ken Massey, Ed Kambor, Pythagorean Ratings


  BEST AGAINST THE SPREAD (Second Half)

Winner:  Updated line
Winner: Sonny Moore

  The second half saw some very good ATS performances. 21 systems them better than 55%.
49 did better than 50% compared to only 6 last season.  The updated line had a 62.22% record.
Sonny Moore was the best of those who had picks across all games, 78-52, 60%.

  Last year about the same number finished better than 60%. For a comparison,
last season the computer adjusted line was 65%.  Despite their name, covers.com finished with a
historically low 36%. The system average was only 40%. Usually trustworthy Sagarin was also only 40%.

Winner 2017: Updated Line 62.222%, Sonny Moore 60%
Winner 2016: Updated Line 54%  ProComputerGambler  53.435%
Winner 2015: Computer adjusted line, Dokter Entropy.
Winner 2014: Least Squares regression, computer adjusted line (record  65.476% in 63% of games)
Winner 2013: Sonshine forecast  63.551%
Winner 2012: Sagarin Predictor, Sagarin Overall
Winner 2011: Kambour Ratings
Winner 2010: Sagarin Predictor, Edward Kambour
Winner 2009: Grid Iron Gold
Winner 2008: Ken Ashby
Winner 2007: Least Squares with Team Specific HFA
Winner 2006: CPA Rankings
Winner 2005: Tom Benson
Winner 2004: ARGH Power Ratings
Winner 2003: Logistic Regression, Viacheslav Ugolnikov
Winner 2002: Least Absolute Value Regression (record against updated line 78-52, 60.0%)
Winner 2001: Stat Fox
Winner 2000: Pythagorean Ratings
Winner 1999: Least squares using team specific home field advantages




  SMALLEST MEAN ABSOLUTE ERROR (Second Half)

Winner: Updated Line

The updated line had the lowest mean error in the seond half, 9.3926.
The best non line based system was Game Time Decision at 9.5406

Winner 2017: Vegas Line (updated)  9.3926
Winner 2016: Vegas Line (updated)  (Record 9.1903)
Winner 2015: Talisman Red
Winner 2014: Least Squares regression
Winner 2013: Ironrank.com
Winner 2012: Lou St. John
Winner 2011: Turnover adjusted least squares
Winner 2010: Massey Ratings
Winner 2009: Vegas Line (updated)
Winner 2008: The Entropy System
Winner 2007: Vegas Line (updated)
Winner 2006: Jeff Self
Winner 2005: Vegas Line (updated)
Winner 2004: Least Squares Regression
Winner 2003: Vegas Line (updated)
Winner 2002: FreeSportsPlays.com
Winner 2001: Vegas Line (updated)
Winner 2000: Pythagorean Ratings




  SMALLEST BIAS (Second Half)    

Winner: teamrankings.com


TeamRankings.com had the smallest bias in the second half at -0.0341.  Over the full season
the biases balanced out around zero but ove the second half they were almost all negative with
a median value of around -0.70.

Winner 2017: Teamrankings.com    -0.03410
Winner 2016: ESPN FPI              .18680
Winner 2015: LS - w/ team HFA
Winner 2014: Locksmith picks
Winner 2013: Covers.com
Winner 2012: Nutshell Sports
Winner 2011: Lee Burdorf
Winner 2010: CPA Retro Rankings
Winner 2009; Game Time Decision
Winner 2008: CPA Rankings
Winner 2007: Pigskin Index (record 0.0077)
Winner 2006: CPA Retro Rankings
Winner 2005: Kasulis Enhanced Spread
Winner 2004: Tom Benson Gridmarks
Winner 2003: CPA Rankings
Winner 2002: StatFox.com
Winner 2001: Monte Carlo Markov Chain
Winner 2000: Flyman Performance Ratings
Winner 1999: Flyman Performance Ratings




  MOST ACCURATE SYSTEM - SMALLEST MEAN SQUARE ERROR (Second Half)

Winner: Updated Line


 The computer adjusted line had the smallest mean square error in the second half, 150.31000.
Fairly close to the record set last season. SportsCruncher apepars to have a lower MSE at
149 but was missing 5 games.  Take those 5 games out for everyone and he falls all the way
down to 7th place. Game Time Decision was the best of the non line systems.


Winner 2017: Updated Line  150.31000
Winner 2016: Computer Adjusted Line (record 147.025)
Winner 2015: Talisman Red
Winner 2014: Ironrank.com
Winner 2013: Steven Jens
Winner 2012: Vegas Line (updated)
Winner 2011: Turnover adjusted least squares
Winner 2010: Jeff Sagarin
Winner 2009: Vegas Line (updated)
Winner 2008: Hank Trexler
Winner 2007: Vegas Line (updated)
Winner 2006: CPA Rankings
Winner 2005: Vegas Line (updated)
Winner 2004: Computer Adjusted Line
Winner 2003: Viacheslav Ugolnikov
Winner 2002: FreeSportsPlay.com
Winner 2001: Hank's Power Ratings
Winner 2000: Pythagorean Ratings




   BEST PREDICTIVE SYSTEM (Entire Season)

Winner: Donchess Inference

To come up with the best overall predictive system I give each system
points for how well they do in all of the above categories.  I then sum up
the points and the system with the highest total is dubbed predictive system
of the year.  So this award goes to the system that is the most well rounded.
One flaw can totally take a system out of the running.
  For those that had pedictions for the full season, or at least most of the
season, the best overall predictive system was the Donchess Inference Index from
Dratings.com.   Donchess was first place in wins, plus was 55.6% ATS as well as
top 10 in accuracy.  Technically it was the updated line that was the overall
winner by a fairly wide margin.  But Donchess was second with nobody else close
behind. Yourlinx was as far behind Donchess as Donchess was behind the line.


Winner 2017: Donchess Inference
Winner 2016: Game Time Decision
Winner 2015: Doktor Entropy
Winner 2014: Pythagorean Ratings
Winner 2013: Ed Bemiss
Winner 2012: Sagarin Pure Points
Winner 2011: nationalsportsrankings.com
Winner 2010: LVSC opening line
Winner 2009: Vegas Line
Winner 2008: Ken Ashby
Winner 2007: Jeff Sagarin
Winner 2006: Brian Gabrielle
Winner 2005: Vegas Line
Winner 2004: Least Absolute Value Regression
Winner 2003: Beck Elo
Winner 2002: Matthews Grid
Winner 2001: CPA Rankings
Winner 2000: Ed Kambour


  BEST PREDICTIVE SYSTEM (Second Half)



Winner: Game Time Decision

  The top point getter in the second half was again the updated line but unlike
the full season the lead was only 3 points better than Game Time Decision. And
GTD had only a 0.1 win over the full season winner, Donchess Inference.

Winner 2017: Game Time Decision
Winner 2016: ESPN FPI
Winner 2015: Talisman Red
Winner 2014: GameTimeDecison
Winner 2013: IronRank.com
Winner 2012: Sagarin Pure Points
Winner 2011: Edward Kambour Ratings
Winner 2010: LVSC opening line
Winner 2009: Game Time Decision
Winner 2008: Dokter Entropy System
Winner 2007: Cover 81
Winner 2006: CPA Retro
Winner 2005: Hank Trexler
Winner 2004: ARGH Power Ratings
Winner 2003: CPA Rankings
Winner 2002: Kenneth Massey
Winner 2001: Monte Carlo Markov Chain
Winner 2000: Pythagorean Ratings






       Totals Winners     


  AGAINST THE SPREAD 

2017 Talismon Red,  54.023%
2016 Opening Line,  56.500%


  Absolute Error 

2017 Midweek Line,   11.2744
2016 Updated Line,    9.8184


  Mean Square Error 

2017 Midweek Line,   207.49800
2016 Updated Line,   151.28300


  Bias 

2017 Cleanup Hitter,       0.17860
2016 ComPughter Ratings,  -0.03090



  AGAINST THE SPREAD Second Half

2017 RP Excel 2,     59.350%
2016 Pi-Ratings,     58.763%


  Absolute Error Second Half 

2017 Midweek Line,   10.8694
2016 RP Excel 2,      9.1278


  Mean Square Error Second Half 

2017 Midweek Line,    196.83200
2016 RP Excel 2,      134.52000


  Bias  Second Half 

2017 Dan Hanson,         0.14380
2016 System Average,     0.29170