Welcome to the 10th annual NCAA PT Awards.  We've completed our first decade.
The purpose of these awards is to help raise the publicity for those systems that have been
superior in various qualities of interest.  Most awards will be based entirely on the numbers
I've monitored with my prediction tracker web pages. For the 2010  season I followed the
weekly performance of 68 computer rating  systems. This is up from 61 last year and 54 the
year before.  There has definitely been a growth spurt in the last couple of years.
What set this year apart is that we had new teams challanging for the national championship.
I don't think many picked Auburn as their preseason #1.




  MOST STRAIGHT UP WINNERS (Entire Season)

Winner: The Entropy System, Jon Dokter



  This year I collected predictions on 718 games involving two 1A teams. (I
don't care what they are now, I am still going to call them 1A.)  This year
Jon Dokter's Entropy System led in most straight up winners iwth a recod of
546-172 (76.045%).  The last two seasons the winner has been in the 75.5 to
75.6 percent range.  This is the highest since 2006 and about a half point
below the record of 76.48 set in 2002.
 This year only three systems got more games correct than the vegas line,
Dokter, Dunkel, and the computer adjusted line, which starts off equal to
the line.  Last season 21 systems were better than the line.


2010 Winner: Dokter Entropy
2009 Winner: Pigskin Index
2008 Winner: System Average, System Median
2007 Winner: StatFox
2006 Winner: Thompson SPRS, system average, Sagarin predictive, & Sagarin overall
2005 Winner: Pigskin Index
2004 Winner: Jeff Sagarin's Overall Rating
2003 Winner: Edward Kambour & Jeff Sagarin's Overall Rating
2002 Winner: ARGH Power Ratings (record 76.48%)
2001 Winner: Massey Ratings
2000 Winner: ARGH Power Ratings
1999 Winner: Vegas line




  SMALLEST DEVIATION FROM ACTUAL GAME SCORES (Entire Season)

Winner:  Computer Adjusted Line

  Deviation from the game score is the difference between the game prediction
and the actual result.  A value of zero would mean the score difference is predicted
exactly.  One property of a good system would be to minimize the system's average
game deviation.
  I'm glad that the my computer adjusted line is proving to be just as good or not
better than the Vegas line.  The computer adjusted line works similarly to the way the
sportsbook adjust line.  It starts off equal to the line but fractions are added or
taken away from the line depending on what proportion of the computer systems are on
one side of the line.  Where one side means being at least 1.5 points away from the
line.  I think the fact that this 'system' is beating the line in this category is
showing that the computers taken as a whole, do add useful information.
 The winning mark this year was 12.4106. That is over a point worse than the record set
last season but is more in line with a typical season.  The line and then the opening
line were second and third. Then the system average and median were 4th and 5th. You
have to look all the way down to 6th place to find the first independent ratings system.
That was Atomic Football.


2010 Winner: Computer Adjusted Line
2009 Winner: Computer Adjusted Line (record 11.26)
2008 Winner: Vegas Line
2007 Winner: Vegas Line
2006 Winner: Computer Adjusted Line
2005 Winner: Vegas Line
2004 Winner: Vegas Line
2003 Winner: Vegas Line
2002 Winner: Vegas Line
2001 Winner: Vegas Line
2000 Winner: Vegas Line
1999 Winner: Vegas Line




  SMALLEST AVERAGE GAME BIAS (Entire Season)

Winner: Sagarin Predictor


  Bias is a little different from deviation.  Deviation measures the
distance between a prediction and the actual result.  Bias combines
distance and location of the prediction.   Bias measures whether the
predictions are too high or too low.  So if a sytem has an
average bias of +0.25 that means that on average the system
gives 0.25 points too many to the home team.
  There were a few systems this year that had very small bias.  The Best was Sagarin's
Predictor or Points ratings, at a very good 0.001.  But Billingsley's points based
system was right behind at 0.0029.  You can't get much better than those.


2010 Winner: Sagarin Predictor
2009 Winner: CPA Retro Rankings
2008 Winner; Opening Line
2007 Winner: Thompson SPRS
2006 Winner: Congrove Computer Rankings
2005 Winner: Computer Adjusted Line
2004 Winner: CPA Rankings
2003 Winner: Edward Kambour
2002 Winner: Gupta Power Ratings
2001 Winner: Flyman
2000 Winner: ARGH Power Ratings
1999 Winner: Jeff Sagarin




  BEST AGAINST THE SPREAD (Entire Season)


Winner: Catherwood Ratings

  Beating the spread is not the number one goal for most computer
rating systems.  But it is something that the average person likes
to look at and probably the most popular subject that I get emails
about.  I know that looking at all the games is probably not the
most meaningful of measurements but it is what I have been reporting.
  This year about a third of the rating systems did bettert than
50% against the spread, but many of those were second half only systems
and only two systems with full season data did better than 51%.
The best this season was Catherwood ratings, 358-313 (53.353%). The
other system over 515 was Brent Craig.  On the flip side, Schmidt Ratings
was only 43.4%.  As is often the case you could have done better betting
against the worst system than bettin with the best system.



2010 Winner: Catherwood Ratings
2009 Winenr: Covers.com
2008 Winner: Stephen Kerns
2007 Winner: Nutshell Sports
2006 Winner: Harmon Forecast  (record 56.4%)
2005 Winner: Keeper's College Football Ratings
2004 Winner: Edward Kambour
2003 Winner: Edward Kambour
2002 Winner: Dunkel Index
2001 Winner: BMC Picks
2000 Winner: Dunkel Index
1999 Winner: Average across all systems



  MOST ACCURATE PREDICTOR (Entire Season)

Winner:  Computer Adjusted Line

  This award is based on mean square error. Mean square error takes
into account both deviation and bias and is perhaps the most commonly
used measure of evaluating estimators.
  Something other than the Vegas line wins this award for the first time
in 2010.  The Vegas line falls to second place and the computer adjusted
line takes the top spot with a mean square error of 244.423.  The updated
line was a point behind no one else was very close but again the closest were
the opening line, system average and median.  The top independent system
was Dokter in 6th place.


2010 Winner: Computer Adjusted Line
2009 Winner: The Vegas Line  (record 206.152)
2008 Winner: The Vegas Line
2007 Winner: The Vegas Line
2006 Winner: The Vegas Line
2005 Winner: The Vegas Line
2004 Winner: The Vegas Line
2003 Winner: The Vegas Line
2002 Winner: The Vegas Line
2001 Winner: The Vegas Line
2000 Winner: The Vegas Line
1999 Winner: The Vegas Line



    ------------------------


  It is debatable how meaningful rating systems are in the early
parts of the season.  How can you rate the teams in the first
week of the season when no teams have played a game yet?  The BCS
waits until mid October before releasing it's first ranking
of teams.  Some systems choose to wait until around this time
before being made public. Looking at only the second half of the
season also gives an estimate of how well a system does based
only(or mostly) on data from the current year.



   MOST STRAIGHT UP WINNERS (Second Half of Season)

Winner: Brent Craig


  The second half data started with week 9 and consisted of 374 games
between two division 1A teams.

Brent Craig's second system had the best record in the second half,
275-99, 73.529%.  Craig finished 3 games better than Doktor.
Interestingly 22 systems did better than the line in over the second
half while only 3 did over the full season.


2010 Winner:  Brent Craig
2009 Winner:  Massey Ratings  (record 77.688%)
2008 Winner:  Nutshell Sports
2007 Winner:  Edward Kambour
2006 Winner:  Ed Bemiss
2005 Winner:  Dunkel Index
2004 Winner:  Jeff Sagain, Wesley Colley
2003 Winner:  Born Power Index & Edward Kambour
2002 Winner:  Born Power Index
2001 Winner:  Chris Montgomery
2000 Winner:  Geoff Freeze




  SMALLEST DEVIATION FROM ACTUAL GAME SCORES  (Second Half of Season)


Winner: Computer Adjusted Line

  As with the full season, the computer adjusted line had the lowest mean absolute error at
12.8928.  This is a point and a half worse than the record set last season.  An honorable
mention goes to nationalsportsrankings, which was the only other system and ony independent
system, to do better than the line.


2010 Winner:  Computer Adjusted Line
2009 Winner:  Computer Adjusted Line (record 11.2917)
2008 Winner:  StatFox
2007 Winner:  Edward Kambour
2006 Winner:  Computer Adjusted Line
2005 Winner:  Vegas Line
2004 Winner:  Ken Ashby AccuRatings
2003 Winner:  Vegas Line (opening)
2002 Winner:  Vegas Line
2001 Winner:  Vegas Line
2000 Winner:  Vegas Line




  SMALLEST AVERAGE GAME BIAS  (Second Half of Season)

Winner:  Sonny Moore Power Ratings

  Sonny Moore had the smallest bias in the second half, 0.0099.  For the full season the bias
averaged about zero but for the second half it leaned towards positive values.  This also
happened last year and then it was just the opposite in the year before.


2010 Winner:  Sonny Moore
2009 Winner:  Nutshell Girl
2008 Winner:  Dunkel Index
2007 Winner:  CPA Rankings
2006 Winner:  Vegas Line
2005 Winner:  Brent Moore
2004 Winner:  Least Squares Regression
2003 Winner:  Darrly Marsee
2002 Winner:  Massey BCS
2001 Winner:  PerformanZ Ratings
2000 Winner:  Darryl Marsee's Rankings




  BEST AGAINST THE SPREAD  (Second Half of Season)


Winner:  Least squares regression

  There were a few systems that had good against the spread performances over the second half.
The best was simple least squares regression, with a record of 206-162, 55.978%.  This is up
a little bit from last season but well below a record.  I am a little surprised at how well
least squares regression works in college football but since I have added it it has been one
of the consistently best overall performers in the second half of the season.


2010 Winner:  Least squares regression
2009 Winner:  Sonny Moore
2008 Winner:  StatFox
2007 Winner:  Nutshell Sports
2006 Winner:  Ed Bemiss
2005 Winner:  The Sports Report
2004 Winner:  StatFox          (record 58.94%)
2003 Winner:  Edward Kambour
2002 Winner:  The Sports Report
2001 Winner:  BMC Picks
2000 Winner:  The Buck System




  MOST ACCURATE PREDICTOR (Second Half of Season)


Winner: Computer Adjusted Line

  The computer adjusted line (266.538) wins this category for the second year in a row
but well off the record it set last season.  As with absolute error, Nationalsportsranking
was the only other system and only independent system to better the line.


2010 Winner:  Computer Adjusted Line
2009 Winner:  Computer Adjusted Line    (record 208.618)
2008 Winner:  Born Power Index
2007 Winner:  Atomic Football
2006 Winner:  Vegas Line
2005 Winner:  Vegas Line
2004 Winner:  Ashby Accuratings
2003 Winner:  Vegas Line
2002 Winner:  Vegas Line
2001 Winner:  Vegas Line
2000 Winner:  Vegas Line




  BEST PREDICTIVE SYSTEM  

Winner: Catherwood Ratings


  To come up with the best overall predictive system I give each system
points for how well they do in all of the above categories.  I then sum up
the points and the system with the highest total is dubbed predictive system
of the year.  So this award goes to the system that is the most well rounded
One flaw can totally take a system out of the running.

This season the top spot goes to relative newcomer Catherwood Ratings.  I have
Catherwood ranked 3rd for the full season and 6th for the second half, but that
adds up to first place overall.  The line and computer adjusted line grade out
the best for the full season while least squares regression graded out the best
for the second half.  I could have given this award to least square regression
but since it is only a second half system I am not sure if it can be justified.
Overall it was Catherwood, then you have to look past the line based ratings,
updated line, opening line and computer adjusted line before you come to Massey
and Dunkel.



2010 Winner:  Catherwood Ratings
2009 Winner:  Ed Kambour
2008 Winenr:  StatFox
2007 Winner:  Harmon Forecast
2006 Winner:  Computer Adjusted Line
2005 Winner:  Dunkel Index
2004 Winner:  Ken Ashby AccuRatings
2003 Winner:  Ed Kambour
2002 Winner:  Ed Kambour
2001 Winner:  Kenneth Massey (non-BCS)
2000 Winner:  ARGH Power Ratings





     RETRODICTION AWARDS 

  Retrodiction refers to 'retrodicting' or postdicting the previous results,
rather than predicting future games.  It is possible that a computer rating
system can put more emphasis on explaining past results than attempting to
predict future results.  So these systems may not neccesarily be the best
predictive systems but can still be very good at their main objective.
  The retrodiction results on my page come from taking the final
ratings and using them to repredict the entire season.


I break the retrodictive categories into what I will call complex and simple systems.
But this may be the final year I do that.  There are not a lot of complex entries any more.
A simple system will be any system that estimates only a global home
field advantage.   Systems that estimate individual home field advantages
are always going to end up winning these awards so this opens up new
categories for the more traditional systems.



  MOST RETRODICTIVE WINS  

Winner: Superlist

 The Waits/Wilson/Huckaby Nutshell Sports Superlist had the most retrodictive wins this season
with a record of 593-125, 82.591%.  This is the second time this composite of systems has
won this award, it also won back in 2006.



2010 Winner:  The SuperList
2009 Winner:  Jeff Sagarin Elo
2008 Winner:  Jeff Sagarin
2007 Winner:  Jeff Sagarin
2006 Winner:  The SuperList
2005 Winner:  Jeff Sagarin, Warren Claassen
2004 Winner:  Logistic Regression   (record 86.0%)
2003 Winner:  Anderson/Hester
2002 Winner:  Logistic Regression
2001 WInner:  System Average




  MOST RETRODICTIVE WINS  -  Complex System

Winner:


2010 Winner:  N/A
2009 Winner:  Least squares with team specific HFA
2008 Winner:  Least squares with team specific HFA
2007 Winner:  The Sports Report - SLOTS
2006 Winner:  The Sports Report - SLOTS
2005 Winner:  The Sports Report - SLOTS
2004 Winner:  The Sports Report - SLOTS   (record 92.78%)
2003 Winner:  The Sports Report - SLOTS
2002 Winner:  The Sports Report - SLOTS
2001 Winner:  CPA Rankings
2000 Winner:  CPA Rankings





  SMALLEST RETRODICTIVE MEAN ERROR  -  Simple System

Winner: Least Squares Regression

   Simple, good old fashioned least squares regression wins this category
for the second year in a row and 4th time in eight years.  The average error
was 10.4561 points.  This was a full point worse than the record set last year.


2010 Winner:  Least Squares Regression
2009 Winner:  Least Squares Regression  (record 9.4972)
2008 Winner:  System Average
2007 Winner:  Jeff Sagarin Predictive
2006 Winner:  Jeff Sagarin Predictive
2005 Winner:  Jeff Sagarin Predictive
2004 Winner:  Least Squares Regression
2003 Winner:  Least Squares Regression





  SMALLEST RETRODICTIVE MEAN ERROR  -  Complex System

Winner:    Least square Regression with team HFA


2010 Winner:  Least square Regression with team HFA
2009 Winner:  Least square Regression with team HFA  (record 8.3487)
2008 Winner:  Least square Regression with team HFA
2007 Winner:  Least Square Regression with team HFA
2006 Winner:  Least Square Regression with team HFA
2005 Winner:  Least Square Regression with team HFA
2004 Winner:  Least Square Regression with team HFA
2003 Winner:  The Sports Report - SLOTS
2002 Winner:  The average of all systems.
2001 Winner:  Edward Kambour Football Ratings
2000 Winner:  CPA Rankings







  SMALLEST RETRODICTIVE BIAS 

Winner: Sagarin Points

 The system that had the smallest bias was Jeff Sagarin's points, or predictive rating system.
 This system had a bias of 0.0069.  All but 9 systems had a positive value.



2010 Winner:  Sagarin Points/predictor
2009 Winner:  Sportrends
2008 Winner:  Sonny Moore
2007 Winner:  Least Squares w/ team HFA
2006 Winner:  Sports Report - SLOTS
2005 Winner:  Sports Report - SLOTS
2004 Winner:  Anderson/Hester
2003 Winner:  Anderson/Hester
2002 Winner:  CPA Rankings
2001 Winner:  CPA Rankings
2000 Winner:  CPA Rankings





 MOST ACCURATE RETRODICTIVE RATING  -  Simple System 

Winner:  Least Squares Regression

  It should not be too surprising to see least squares regression do well
here since that is what it does, minimize the square errors.  Least squares
ended with a mean square error of 172.505.


2010 Winner:  Least Squares Regression
2009 Winner:  Least Squares Regression  (record 148.888)
2008 Winner:  Least Squares Regression
2007 Winner:  Least Squares Regression
2006 Winner:  Least Squares Regression
2005 Winner:  Least Squares Regression
2004 Winner:  Least Squares Regression
2003 Winner:  Least Squares Regression




 MOST ACCURATE RETRODICTIVE RATING  -  Complex System 


Winner:  Least Square Regression with team HFA

2010 Winner:  Least Square Regression with team HFA
2009 Winner:  Least Square Regression with team HFA  (record 120.486)
2008 Winner:  Least Square Regression with team HFA
2007 Winner:  Least Square Regression with team HFA
2006 Winner:  Least Square Regression with team HFA
2005 Winner:  Least Square Regression with team HFA
2004 Winner:  Least Square Regression with team HFA
2003 Winner:  The Sports Report - SLOTS
2002 Winner:  CPA Rankings
2001 Winner:  Edward Kambour FOotball Ratings





BEST RETRODICTIVE SYSTEM  -  Simple system 

Winner: Jeff Sagarin


This comes as somewhat of a suprise to me because I didn't see it coming. The
best all around retrodictive system this year was Jeff Sagarin.  Last year's winner
Payne, wasn't too far behind.


2010 Winner:  Jeff Sagarin
2009 Winner:  Payne College Football Power Ratings
2008 Winner:  System median
2007 Winner:  Least Squares
2006 Winner:  Warren Claassen
2006 Winner:  Warren Claassen
2005 Winner:  Peter Wolfe
2004 Winner:  Frank Alder
2003 Winner:  Anderson & Hester


BEST RETRODICTIVE SYSTEM -  Complex System 

For now this award is being retired with Least Square Regression with team HFA
and TSR SLOTS being enshrined into the retrodictive ratings hall of fame.
Unless something else comes along that can compete I am not going to bother
giving every award to Least Square Regression with team HFA.

2009 Winner:  Least Square Regression with team HFA
2008 Winner:  Least Square Regression with team HFA
2007 WInner:  TSR SLOTS
2006 WInner:  TSR SLOTS
2005 WInner:  TSR SLOTS
2004 Winner:  CPA Rankings
2003 WInner:  TSR SLOTS
2002 Winner:  CPA Rankings
2001 Winner:  CPA Rankings
2000 Winner:  CPA Rankings