Welcome to the 16th annual NFL Prediction Tracker Awards. The purpose of these awards is
to help raise the publicity for those systems that have been superior in various qualities of
interest. The awards will be based entirely on the numbers I've monitored with my prediction
tracker web pages. In 2015 I tracked 69 rating/pediction systems that had data in he NFL for
at least half of the season. This was down 3 from the prior season.


BEST STRAIGHT UP WINNERS (Entire Season)

Winner: Sagarin Golden Mean
Winner: YourLinx

straight up plot

 We had a tie for best straight up record this season between Sagarin Golden Mean and
Yourlinx. Both had a record of 176-91, 55.985%  This is one of the lower winning totals
that we have seen. It matches 2010 and the last time we saw a winner with a lower record
was 2006.  The winner last year, Dokter Entropy, and Daniel Curry Index finished one game
behind.  The updated line, finished in 32nd place, near the middle of the pack.  That
is two years in a row that the line has been a relatively poor predictor of winners.
The updated line did 4 games better than the opening line which did 4 games better than the
midweek line.
 From the plot we see the leaders were around the top the whole season. And it appears that
the winning percentages stayed fairly stable through the season.


Winner 2015: Sagarin Golden Mean, YourLinx
Winner 2014: Dokter Entropy
Winner 2013: Ed Bemiss
Winner 2012: Sagarin Pure Points (predictive)
Winner 2011: Nationalsportsranking.com
Winner 2010: LVSC opening line
Winner 2009: JFM Power Ratings, Sportrends
Winner 2008: Ken Ashby
Winner 2007: Sonny Moore
Winner 2006: Least Absolute Value Regression
Winner 2005: Vegas Line (record 72.28%)
Winner 2004: Least Absolute Value Regression
Winner 2003: Beck Elo
Winner 2002: Herman Matthews
Winner 2001: JFM Power Ratings, ARGH Power Ratings
Winner 2000: CPA Rankings
Winner 1999: Kenneth Massey Ratings




  BEST AGAINST THE SPREAD (Entire Season)

Winner: Logistic Regression

ATS plot

 Some consider the true worth of a ranking system to be its record against the Vegas line.
Although that record for every game is probably not the best measurement but that is what I have.
This year Logistic Regression lead the season by setting a new all time record, 152-103,
59.608%.  The old record was set by Lou St. John in 2013.
 It is interesting that a system which uses the schedule with nothing but who won and
who lost would be a system that sets a new ATS record.  Computer Adjusted Line, almost always
near the top, finished it second.  Dokter Entropy and RP Excel also finished above 58% and would
have been good enough to win last year. In general systems did better then 50%, the median
performance was about 52.9%


Winner 2015 Logistic Regression (record 59.608%)
Winner 2014: Computer adjusted line, Turnover adjusted least squares regresssion
Winner 2013: Lou St. John  (record 59.449%)
Winner 2012: Least squares regression
Winner 2011: Pi-Rate Ratings
Winner 2010: Sagarin Predictor
Winner 2009: CPA Rankings.
Winner 2008: Nutshell Retro, Entropy System
Winner 2007: Sagarin Predictive
Winner 2006: Brian Gabrielle (record against updated line 59.0%)
Winner 2005: Least squares with team HFA
Winner 2004: Least Absolute Value Regression
Winner 2003: CPA Retro Rankings
Winner 2002: FreeSportsPlay.com
Winner 2001: CPA Rankings
Winner 2000: Yourlinx
Winner 1999: PerformanZ Ratings





  SMALLEST MEAN ABSOLUTE ERROR (Entire Season) 

Winner: Updated Line

ATS plot


  Deviation from the game score is the absoulte difference between the game prediction
and the actual result.  A value of zero would mean the score difference is predicted
exactly.  One property of a good system would be to minimize the system's average
game deviation.
  After a few years of not being on the top the updated line returns as champion with
an average error of 10.14 points.  It is kind of mind boggling how Logistic regression,
as was mentioned above, uses nothing but an outcome of win or lose, came in second, only
0.02 points behind the line and ahead of the 3rd and 4th place coputer adjusted line and
opening line.  I hate to say it but this supports my theory that the performance of these
systems are basically random in all of these categories.


Winner 2015: Vegas Line (updated)
Winner 2014: Massey Ratings
Winner 2013: Ed Bemiss (record 9.7236)
Winner 2012: Vegas Line (updated)
Winner 2011: Dokter's Entropy System
Winner 2010: LVSC Opening line
Winner 2009: Vegas Line (updated)
Winner 2008: Ken Ashby
Winner 2007: Computer Adjusted Line
Winner 2006: Brian Gabrielle
Winner 2005: Vegas Line (updated)
Winner 2004: Vegas Line (updated)
Winner 2003: Vegas Line (updated)
Winner 2002: FreeSportsPlay.com
Winner 2001: Vegas Line (updated)
Winner 2000: Vegas Line (updated)
Winner 1999: Vegas Line (updated)





 SMALLEST BIAS (Entire Season)  

Winner: Lou St. John

ATS plot

  Bias is a little different from deviation.  Deviation measures the distance
between a prediction and the actual result.  Bias combines distance and location
of the prediction.   Bias measures whether the predictions are too high or too low.
So if a system has an average bias of +0.25 that means that on average the system
gave 0.25 points too many to the home team.
  Lou St. John had the smallest bias this season at +0.01 points per game.
Massey, Turnover adjusted regression and CPA Retro were the only other systems
under 0.1.  The majority of the systems were positive in the range of .5 to 1
points per game.  For the last several years the typical numbers have been negative.


Winner 2015: Lou St. John
Winner 2014: Sagarin Points Elo
Winner 2013: Game Time Decision
Winner 2012: PerformanZ Ratings
Winner 2011; Lee Burdorf
Winner 2010: Stephen Kerns
Winner 2009; System Median
Winner 2008: ARGH Power Ratings
Winner 2007: CPA Rankings
Winner 2006: Tom Benson
Winner 2005: Sagarin Elo (record 0.001)
Winner 2004: System Average
Winner 2003: CPA Rankings
Winner 2002: Sagarin Pure Points
Winner 2001: ARGH Power Ratings
Winner 2000: Pigskin Index
Winner 1999: Flyman Ratings




 MOST ACCURATE SYSTEM - SMALLEST MEAN SQUARE ERROR (Entire Season)


Winner: Dokter Entropy

ATS plot

 This award is based on mean square error. Mean square error takes
into account both deviation and bias and is a commonly used measure
of evaluating estimators.
Dokter Entropy, of TimeTravelSports.com had the lowest mean square error for the 2015
season with a fairly good number of 167.21. The win was by a narrow margin of 0.3 over
the updated line and computer adjusted line.  Dokter has won the category once before
in 2011 and is one of only two non-line systems to win over the last 13 seasons.


Winner 2015: Dokter Entropy
Winner 2014: Computer Adjusted Line
Winner 2013: Lou St. John (record 157.3140)
Winner 2012: Vegas Line (updated)
Winner 2011: Dokter Entropy System
Winner 2010: Computer Adjusted Line
Winner 2009: Vegas Line (updated)
Winner 2008: Computer Adjusted Line
Winner 2007: Computer Adjusted Line
Winner 2006: Computer Adjusted Line
Winner 2005: Vegas Line (updated)
Winner 2004: Vegas Line (updated)
Winner 2003: Vegas Line (opening)
Winner 2002: FreeSportsPlay.com
Winner 2001: Vegas Line
Winner 2000: Pigskin Index




            SECOND HALF AWARDS 


Some systems donít begin publishing rankings until every team has played a couple of games, while
the season-long systems often rely on preset starting values in the early part of the season. For
those reasons, I like to look at the results over the second half of the season when these systems
have had time to become 'burned in' to  the season's data. Second half data consists of all games
from week 10 through the Super Bowl.



  BEST STRAIGHT UP WINNERS (Second Half)

Winner: Talisman Red

straight up plot

  Talisman Red had  the most wins in the second half, 88-47, 65.185%.  This is not really a good
winning mark. Last year for example, Logistic regression was first place with 72%, or 9 games better.
Nutshell Middle Game was 1 game back in second place.
Looking at the plot the second half got off to a terrible start in week 10, then gradually settled in
with an average around 60%. It also shows that Talisman took over in week 11 and never trailed again.
The updated line really hasn't performed that well straight up the last few seasons. It finished in
28th place this season.


Winner 2015: Talisman Red
Winner 2014: Logistic Regression
Winner 2013: Ed Bemiss
Winner 2012: Sonny Moore, Kasulis Enhanced Spread
Winner 2011: Turnover adjusted least squares
Winner 2010: LVSC opening line
Winner 2009: The Vegas line, Computer adjusted line, Ashby AccuRatings, StatFox, Upset Report, CPA Rankings
Winner 2008: Nutshell Girl
Winner 2007: Computer Adjusted Line, Vegas Line (updated)
Winner 2006: Least Absolute Value Regression
Winner 2005: Vegas Line (updated), Nutshell Sports (record 100-37, 72.99%)
Winner 2004: Pigskin Index
Winner 2003: JFM Power Ratings, Kasulis Enhanced Spread, Beck Elo
Winner 2002: Least Absolute Value Regression, JFM Power Ratings, CPA Rankings
Winner 2001: Pythagorean Ratings, Elo Ratings, Ken Massey Ratings
Winner 2000: PerformanZ Ratings, Scoring Effeciency
Winner 1999: Ken Massey, Ed Kambor, Pythagorean Ratings




  BEST AGAINST THE SPREAD (Second Half)

Winner:  Computer Adjusted line
Winner: Dokter Entropy

straight up plot

  The computer adjusted line had the best ATS record in the second half, 54-31, 63.5%
This system has had a great two year run doing 65.4% last year.  But since the computer
adjusted line is equal to the line about a third of the time by design I like to name
a co-winner that picked all the games.  This year the co-winner is Dokter Entropy, 80-51,
61.069%  There were 4 other systems that finished better than 60%, so this was a fairly
easy year ATS in the second half with the average system winning almost 54% of the time.
Only 6 full time participants finished less than 50%.  These good performances are almost
identicle to what happened last year.


Winner 2015: Computer adjusted line, Dokter Entropy.
Winner 2014: Least Squares regression, computer adjusted line (record  65.476% in 63% of games)
Winner 2013: Sonshine forecast (record 68-39, 63.551%)
Winner 2012: Sagarin Predictor, Sagarin Overall
Winner 2011: Kambour Ratings
Winner 2010: Sagarin Predictor, Edward Kambour
Winner 2009: Grid Iron Gold
Winner 2008: Ken Ashby
Winner 2007: Least Squares with Team Specific HFA
Winner 2006: CPA Rankings
Winner 2005: Tom Benson
Winner 2004: ARGH Power Ratings
Winner 2003: Logistic Regression, Viacheslav Ugolnikov
Winner 2002: Least Absolute Value Regression (record against updated line 78-52, 60.0%)
Winner 2001: Stat Fox
Winner 2000: Pythagorean Ratings
Winner 1999: Least squares using team specific home field advantages





  SMALLEST MEAN ABSOLUTE ERROR (Second Half)

Winner: Talisman Red

straight up plot

Talisman Red had the lowest mean error in the seond half, 10.6754. This is about the same
as the winner last year but almost a point off the record setting number.  You can see from
the graph that the numbers were skewed higher in large part due to poor performances in weeks
10 and 14. The line  must have dropped off, it managed to finish first for the full season but
was only 8th in the second half.


Winner 2015: Talisman Red
Winner 2014: Least Squares regression
Winner 2013: Ironrank.com (record  9.7386)
Winner 2012: Lou St. John
Winner 2011: Turnover adjusted least squares
Winner 2010: Massey Ratings
Winner 2009: Vegas Line (updated)
Winner 2008: The Entropy System
Winner 2007: Vegas Line (updated)
Winner 2006: Jeff Self
Winner 2005: Vegas Line (updated)
Winner 2004: Least Squares Regression
Winner 2003: Vegas Line (updated)
Winner 2002: FreeSportsPlays.com
Winner 2001: Vegas Line (updated)
Winner 2000: Pythagorean Ratings



  SMALLEST BIAS (Second Half)    

Winner: Least Square Regression with team specific HFA

straight up plot

 The version of least squares regression that incorporated team specific home field advantage
estimates has the lowest bias in the second half, 0.06230.  The numbers were not very good at
all, CPA was the only other system under 0.25.  The average was about 1.4.
The numbers started out really bad in week 10 and then tried to recover the rest of the season.


Winner 2015: LS - w/ team HFA
Winner 2014: Locksmith picks
Winner 2013: Covers.com
Winner 2012: Nutshell Sports
Winner 2011: Lee Burdorf
Winner 2010: CPA Retro Rankings
Winner 2009; Game Time Decision
Winner 2008: CPA Rankings
Winner 2007: Pigskin Index (record 0.0077)
Winner 2006: CPA Retro Rankings
Winner 2005: Kasulis Enhanced Spread
Winner 2004: Tom Benson Gridmarks
Winner 2003: CPA Rankings
Winner 2002: StatFox.com
Winner 2001: Monte Carlo Markov Chain
Winner 2000: Flyman Performance Ratings
Winner 1999: Flyman Performance Ratings




  MOST ACCURATE SYSTEM - SMALLEST MEAN SQUARE ERROR (Second Half)


Winner: Talisman Red

straight up plot

  Talisman Red finished in first place in 3 of the 5 second half categories and finished top
8 in the other two.  That is one of the most impressive second halfs we have ever seen.
He led mean square error with and average of 193.26000.  The updated line finished in second place about
4 points behind.


Winner 2015: Talisman Red
Winner 2014: Ironrank.com
Winner 2013: Steven Jens (record  158.0850)
Winner 2012: Vegas Line (updated)
Winner 2011: Turnover adjusted least squares
Winner 2010: Jeff Sagarin
Winner 2009: Vegas Line (updated)
Winner 2008: Hank Trexler
Winner 2007: Vegas Line (updated)
Winner 2006: CPA Rankings
Winner 2005: Vegas Line (updated)
Winner 2004: Computer Adjusted Line
Winner 2003: Viacheslav Ugolnikov
Winner 2002: FreeSportsPlay.com
Winner 2001: Hank's Power Ratings
Winner 2000: Pythagorean Ratings






   BEST PREDICTIVE SYSTEM (Entire Season)

Winner: Dokter Entropy

To come up with the best overall predictive system I give each system
points for how well they do in all of the above categories.  I then sum up
the points and the system with the highest total is dubbed predictive system
of the year.  So this award goes to the system that is the most well rounded.
One flaw can totally take a system out of the running.
  For those that had pedictions for the full season, or at least most of the
season, the best overall predictive system was Doktor Entropy of Timetravelsports.com.
Doktor had an extremely well rounded season, finishing top 5 in every category except
for bias.  Logistic Regression was a close second and Sagarin Golden mean was
third.


Winner 2015: Doktor Entropy
Winner 2014: Pythagorean Ratings
Winner 2013: Ed Bemiss
Winner 2012: Sagarin Pure Points
Winner 2011: nationalsportsrankings.com
Winner 2010: LVSC opening line
Winner 2009: Vegas Line
Winner 2008: Ken Ashby
Winner 2007: Jeff Sagarin
Winner 2006: Brian Gabrielle
Winner 2005: Vegas Line
Winner 2004: Least Absolute Value Regression
Winner 2003: Beck Elo
Winner 2002: Matthews Grid
Winner 2001: CPA Rankings
Winner 2000: Ed Kambour


  BEST PREDICTIVE SYSTEM (Second Half)



Winner: Talisman Red

  The top point getter inthe second half was Talismanred.com which came close to
doubling the number of points of everyone except second place logistic regression.
Talisman finished with 3 first place finishes and in the top 8 in the other two.


Winner 2015: Talisman Red
Winner 2014: GameTimeDecison
Winner 2013: IronRank.com
Winner 2012: Sagarin Pure Points
Winner 2011: Edward Kambour Ratings
Winner 2010: LVSC opening line
Winner 2009: Game Time Decision
Winner 2008: Dokter Entropy System
Winner 2007: Cover 81
Winner 2006: CPA Retro
Winner 2005: Hank Trexler
Winner 2004: ARGH Power Ratings
Winner 2003: CPA Rankings
Winner 2002: Kenneth Massey
Winner 2001: Monte Carlo Markov Chain
Winner 2000: Pythagorean Ratings