Welcome to the 9th annual NCAA PT Awards.  Hopefully you don't mind that they look
about the same every year.  And I have these out in February, the earliest in a long time.
The purpose of these awards is to help raise the publicity for those systems that have been
superior in various qualities of interest.  Most awards will be based entirely on the numbers
I've monitored with my prediction tracker  web pages. For the 2009  season I followed the
weekly performance of 61 computer rating  systems. This is up from 54 last year.
  It was a somewhat interesting year with so many undefeated teams.  But I hate the way
it ended.  I wish Colt McCoy had not gotten hurt on the first drive and we could have seen
a real championship game.  A few minutes into the game a month of game planning by both
teams was thrown out the window.




  MOST STRAIGHT UP WINNERS (Entire Season)

Winner: Pigskin Index



  This year I collected predictions on 714 games involving two 1A teams. (I
don't care what they are now, I am still going to call them 1A.)  This year
Gary Hawley's Pigskin Index led the way with a 540-174 record, 75.63%
This is about the same percentage it took to win last year 75.5%. This is
a percentage point below the record set by ARGH in 2002.
  Last year I said it was a bad year for the odds makers because 11 teams out
performed the line, but this year 21 systems had a better record than the line.
Vegas, and my lines come from the Vegas Hilton, tends to be in the 73-74%
accuracy range from year to year.  Last years winner, the system average and
honorable mention, Atomic football, both tied for second this year, three games
behind Pigskin Index.  Sagarin had a great run from 2003 to 2006 but has not
challenged the last few years.  For the last five years the systems to most
consistantly pick winners have been the system average, Pigskin Index, and Jim
Ahsburn's Atomic Football.



2009 Winner: Pigskin Index
2008 Winner: System Average, System Median
2007 Winner: StatFox
2006 Winner: Thompson SPRS, system average, Sagarin predictive, & Sagarin overall
2005 Winner: Pigskin Index
2004 Winner: Jeff Sagarin's Overall Rating
2003 Winner: Edward Kambour & Jeff Sagarin's Overall Rating
2002 Winner: ARGH Power Ratings (record 76.48%)
2001 Winner: Massey Ratings
2000 Winner: ARGH Power Ratings
1999 Winner: Vegas line




  SMALLEST DEVIATION FROM ACTUAL GAME SCORES (Entire Season)

Winner:  Computer Adjusted Line

  Deviation from the game score is the difference between the game prediction
and the actual result.  A value of zero would mean the score difference is predicted
exactly.  One property of a good system would be to minimize the system's average
game deviation.
  I never noticed how different this year was when it came to absolute errors.
18 systems beat the previous all time record of 11.81.  That is a phenomenal number.
The computer adjusted line edged out the Vegas line 11.2612 to 11.2654.  What is the
computer adjusted line.  It is a method I came up to make small changed to the Vegas
line based on what proportion of the systems fall certain distances away from the line.
The best manned system this year was John Dokter's Entropy system in 4th place.


2009 Winner: Computer Adjusted Line (record 11.26)
2008 Winner: Vegas Line
2007 Winner: Vegas Line
2006 Winner: Computer Adjusted Line
2005 Winner: Vegas Line
2004 Winner: Vegas Line
2003 Winner: Vegas Line
2002 Winner: Vegas Line
2001 Winner: Vegas Line
2000 Winner: Vegas Line
1999 Winner: Vegas Line




  SMALLEST AVERAGE GAME BIAS (Entire Season)

Winner: CPA Retro Rankings


  Bias is a little different from deviation.  Deviation measures the
distance between a prediction and the actual result.  Bias combines
distance and location of the prediction.   Bias measures whether the
predictions are too high or too low.  So if a sytem has an
average bias of +0.25 that means that on average the system
gives 0.25 points too many to the home team.
  We had a fairly wide range of biases this year.  On average we over
estimated in favor of the home team.  There were a few systems this year
that had very small bias.  The Best was Steve Wrathell's CPA Retro Rankings
with a bias of -0.0054.  This just barely beats Billingsley by 0.0005 points.


2009 Winner: CPA Retro Rankings
2008 Winner; Opening Line
2007 Winner: Thompson SPRS
2006 Winner: Congrove Computer Rankings
2005 Winner: Computer Adjusted Line
2004 Winner: CPA Rankings
2003 Winner: Edward Kambour
2002 Winner: Gupta Power Ratings
2001 Winner: Flyman
2000 Winner: ARGH Power Ratings
1999 Winner: Jeff Sagarin




  BEST AGAINST THE SPREAD (Entire Season)


Winner: 

  Beating the spread is not the number one goal for most computer
rating systems.  But it is something that the average person likes
to look at and probably the most popular subject that I get emails
about.  I know that looking at all the games is probably not the
most meaningful of measurements but it is what I have been reporting.
  Last year the line seemed to have a somewhat poor year for accuracy
but few systems did better than 50% against the spread.  This year
quite a few systems challenged the accuracy of the lines and I would
say about 60% of the systems did better than 50% against the line.
But they just beat the 50-50 level.  Every system that picked for the
full season was in the 47%-53% range.  So there were not any spectacular
performances.  The winner was Covers.com at 53.488%, 368-320.


2009 Winenr: Covers.com
2008 Winner: Stephen Kerns
2007 Winner: Nutshell Sports
2006 Winner: Harmon Forecast  (record 56.4%)
2005 Winner: Keeper's College Football Ratings
2004 Winner: Edward Kambour
2003 Winner: Edward Kambour
2002 Winner: Dunkel Index
2001 Winner: BMC Picks
2000 Winner: Dunkel Index
1999 Winner: Average across all systems



  MOST ACCURATE PREDICTOR (Entire Season)

Winner:  The Vegas line

  This award is based on mean square error. Mean square error takes
into account both deviation and bias and is perhaps the most commonly
used measure of evaluating estimators.
  After two years of declining numbers we jump back to set a new record
in this category. Nine entries were better this year than previous record
of 220.556 set in 2006.   Vegas wins again with a new record low, 206.152.
The computer adjusted line challenged at 206.163 but no others were very
close.  The honorable mention manned system goes to Dokter's Entropy rating
in 4th place.


2009 Winner: The Vegas Line  (record 206.152)
2008 Winner: The Vegas Line
2007 Winner: The Vegas Line
2006 Winner: The Vegas Line
2005 Winner: The Vegas Line
2004 Winner: The Vegas Line
2003 Winner: The Vegas Line
2002 Winner: The Vegas Line
2001 Winner: The Vegas Line
2000 Winner: The Vegas Line
1999 Winner: The Vegas Line



    ------------------------


  It is debatable how meaningful rating systems are in the early
parts of the season.  How can you rate the teams in the first
week of the season when no teams have played a game yet?  The BCS
waits until mid October before releasing it's first ranking
of teams.  Some systems choose to wait until around this time
before being made public. Looking at only the second half of the
season also gives an estimate of how well a system does based
only(or mostly) on data from the current year.



   MOST STRAIGHT UP WINNERS (Second Half of Season)

Winner: Massey Ratings


  The second half data started with week 9 and consisted of 372 games
between two division 1A teams.
  The winner this year is an interesting one because it wasn't a real
system at all but a mistake.  Each year I include all of the BCS system
in the predictions whether they make predictions or not.  Even if they are
not scaled to make predictions they can always be put onto a scale to allow
for predictions.  This generally produces very good retro fitting results but
not as good predictive results.  So that is what I thought I was doing with
Massey's BCS ratings this season up until about two weeks before the end of
the season when it was pointed out to me that his predictions were too good to
be coming from his BCS ratings.  Sure enough I had been taking his margin of
victory ratings and then rescaling them.  Since I had been doing this almost
to the end of the season it did not make sense to change to the real BCS ratings
for just the final weeks, so I left it as they were  The results were spectacular,
a new record 77.688% accuracy in the second half, 289-83.  So now the question
becomes were they so good because of his published ratings or my transformation?
His ratings are pretty good so I will leave the credit with him.  But next year
I will hopefully remember to make his margin of victory and BCS ratings separate
entries.  The old record of 76.7% was set by Born and Kambor in 2003.
Kambour came in second this year, coming close but not passing his old record.



2009 Winner:  Massey Ratings
2008 Winner:  Nutshell Sports
2007 Winner:  Edward Kambour
2006 Winner:  Ed Bemiss
2005 Winner:  Dunkel Index
2004 Winner:  Jeff Sagain, Wesley Colley
2003 Winner:  Born Power Index & Edward Kambour (record 76.7%)
2002 Winner:  Born Power Index
2001 Winner:  Chris Montgomery
2000 Winner:  Geoff Freeze




  SMALLEST DEVIATION FROM ACTUAL GAME SCORES  (Second Half of Season)


Winner: Computer Adjusted Line

  As with the full season, the computer adjusted line set a new record in this
category, 11.2917.  17 systems were lower then the previousrecord of 11.73 set in 2006.
I suspect that the records set in this category  are going to hold up for a long time.
Compare to last season win the winning mark was 12.62.  Honorable mention goes
to Ashburn's Atomic Football, coming in 3rd place behind the line in 2nd.



2009 Winner:  Computer Adjusted Line (record 11.2917)
2008 Winner:  StatFox
2007 Winner:  Edward Kambour
2006 Winner:  Computer Adjusted Line (record 11.73)
2005 Winner:  Vegas Line
2004 Winner:  Ken Ashby AccuRatings
2003 Winner:  Vegas Line (opening)
2002 Winner:  Vegas Line
2001 Winner:  Vegas Line
2000 Winner:  Vegas Line




  SMALLEST AVERAGE GAME BIAS  (Second Half of Season)

Winner:  Nutshell Girl

  The results in this category are interesting.  For the full season the bias averaged around
zeror, for the second half of the season only seven systems had a negative bias.  We saw almost
the exact opposite of last season when only two systems had a positive bias over the second half.
This type of phenomenom is one of the things that I have noticed by doing this over the years.
One of the forgotten variables, and perhaps the most important variable to the accuracy of a
computer ratings, is what I call the underlying environment of the lines.  Sometimes the lines
are consistantly set too high or too low.  I think this is do to random fluctuations in schedules
and outcomes.  My belief in handicapping is that if you can just recognize the environment each
year you have won the battle, then it is just a matter of knowing what type of system performs
best in that environment.
  The smalled average bias for the second half was Nutshell Girl, with -0.0396.


2009 Winner:  Nutshell Girl
2008 Winner:  Dunkel Index
2007 Winner:  CPA Rankings
2006 Winner:  Vegas Line
2005 Winner:  Brent Moore
2004 Winner:  Least Squares Regression
2003 Winner:  Darrly Marsee
2002 Winner:  Massey BCS
2001 Winner:  PerformanZ Ratings
2000 Winner:  Darryl Marsee's Rankings




  BEST AGAINST THE SPREAD  (Second Half of Season)


Winner: Sonny Moore's Computer Power Rankings

  Going back to my comments in the bias category.  Like last year the biases were very heavy in one
direction over the second half, and like last year that led performances against the spread being
a little better than average.  Sonny Moore had a decent record of 199-162, 55.125% over the second half.
But still well below the record of 58.94%



2009 Winner:  Sonny Moore
2008 Winner:  StatFox
2007 Winner:  Nutshell Sports
2006 Winner:  Ed Bemiss
2005 Winner:  The Sports Report
2004 Winner:  StatFox          (record 58.94%)
2003 Winner:  Edward Kambour
2002 Winner:  The Sports Report
2001 Winner:  BMC Picks
2000 Winner:  The Buck System




  MOST ACCURATE PREDICTOR (Second Half of Season)


Winner: Computer Adjusted Line

  The computer adjusted line wins this category with a new record, 208.618.
The opening line and updated line also broke the previous record of 215.55 set
in 2006.  Honorable mention for best manned system is Dokter in 4th place overall.


2009 Winner:  Computer Adjusted Line    (record 208.618)
2008 Winner:  Born Power Index
2007 Winner:  Atomic Football
2006 Winner:  Vegas Line       (record 215.55)
2005 Winner:  Vegas Line
2004 Winner:  Ashby Accuratings
2003 Winner:  Vegas Line
2002 Winner:  Vegas Line
2001 Winner:  Vegas Line
2000 Winner:  Vegas Line




  BEST PREDICTIVE SYSTEM  

Winner: The Dr. Edward Kambour NCAA Football Ratings


  To come up with the best overall predictive system I give each system
points for how well they do in all of the above categories.  I then sum up
the points and the system with the highest total is dubbed predictive system
of the year.  So this award goes to the system that is the most well rounded
One flaw can totally take a system out of the running.

Ed Kambour, who is an old classmate of mine in the Texas A&M statistics department
wins the award for best predictive system of the year for the third time.  He
had once won twice in a row in 2002 & 2003.  Breaking it down, Kambour was 3rd in
the overall season and 1st in the second half,  those were high enough to carry
him to the top.  Sonny Moore was a relatively close 2nd place.



2009 Winner:  Ed Kambour
2008 Winenr:  StatFox
2007 Winner:  Harmon Forecast
2006 Winner:  Computer Adjusted Line
2005 Winner:  Dunkel Index
2004 Winner:  Ken Ashby AccuRatings
2003 Winner:  Ed Kambour
2002 Winner:  Ed Kambour
2001 Winner:  Kenneth Massey (non-BCS)
2000 Winner:  ARGH Power Ratings





     RETRODICTION AWARDS 

  Retrodiction refers to 'retrodicting' or postdicting the previous results,
rather than predicting future games.  It is possible that a computer rating
system can put more emphasis on explaining past results than attempting to
predict future results.  So these systems may not neccesarily be the best
predictive systems but can still be very good at their main objective.
  The retrodiction results on my page come from taking the final
ratings and using them to repredict the entire season.


I break the retrodictive categories into what I will call complex and simple systems.
But this may be the final year I do that.  There are not a lot of complex entries any more.
A simple system will be any system that estimates only a global home
field advantage.   Systems that estimate individual home field advantages
are always going to end up winning these awards so this opens up new
categories for the more traditional systems.


  MOST RETRODICTIVE WINS  -  Complex System

Winner:  Least squares with team specific HFA

 Least squares regression with team a team specific home field advantage had the
best record 602-112 (84.31%)


2009 Winner:  Least squares with team specific HFA
2008 Winner:  Least squares with team specific HFA
2007 Winner:  The Sports Report - SLOTS
2006 Winner:  The Sports Report - SLOTS
2005 Winner:  The Sports Report - SLOTS
2004 Winner:  The Sports Report - SLOTS   (record 92.78%)
2003 Winner:  The Sports Report - SLOTS
2002 Winner:  The Sports Report - SLOTS
2001 Winner:  CPA Rankings
2000 Winner:  CPA Rankings




  MOST RETRODICTIVE WINS  -  Simple System

Winner: Jeff Sagarin

  Jeff Sagarin's elo rating takes over from his main rating as winner in this category.
Sagarin Elo was 600-114, 84.03%.  This is the system that is used in the BCS.
At least one of Sagarin's ratings has won this 4 of the last 5 years.


2009 Winner:  Jeff Sagarin Elo
2008 Winner:  Jeff Sagarin
2007 Winner:  Jeff Sagarin
2006 Winner:  The SuperList
2005 Winner:  Jeff Sagarin, Warren Claassen
2004 Winner:  Logistic Regression   (record 86.0%)
2003 Winner:  Anderson/Hester
2002 Winner:  Logistic Regression
2001 WInner:  System Average





  SMALLEST RETRODICTIVE MEAN ERROR  -  Complex System

Winner:   Least Square Regression with team HFA

  Least squares regression with individual team HFA finishes in first
place for the 6th year in a row with a new record.  Shaving .81 points
off the previous record set in 2006.  Ed Kambour came in second place
and also beat the previous record.


2009 Winner:  Least square Regression with team HFA  (record 8.3487)
2008 Winner:  Least square Regression with team HFA
2007 Winner:  Least Square Regression with team HFA
2006 Winner:  Least Square Regression with team HFA
2005 Winner:  Least Square Regression with team HFA
2004 Winner:  Least Square Regression with team HFA
2003 Winner:  The Sports Report - SLOTS
2002 Winner:  The average of all systems.
2001 Winner:  Edward Kambour Football Ratings
2000 Winner:  CPA Rankings




  SMALLEST RETRODICTIVE MEAN ERROR  -  Simple System

Winner: System Average

   Simple, good old fashioned least squares regression wins this category
again after Sagarin took over for a while.  This year also set a new record,
about a half point below the old record from 2005.  Around 15 systems bettered
the previous record.


2009 Winner:  Least Squares Regression  (record 9.4972)
2008 Winner:  System Average
2007 Winner:  Jeff Sagarin Predictive
2006 Winner:  Jeff Sagarin Predictive
2005 Winner:  Jeff Sagarin Predictive
2004 Winner:  Least Squares Regression
2003 Winner:  Least Squares Regression





  SMALLEST RETRODICTIVE BIAS 

Winner: Sportrends

 I am not breaking this one down into sub categories.  The system that had the smallest
mean bias Sportrends.  This system had a bias of -0.0655.  That is relatively high for
a winner.  All but 6 systems had positive values.



2009 Winner:  Sportrends
2008 Winner:  Sonny Moore
2007 Winner:  Least Squares w/ team HFA
2006 Winner:  Sports Report - SLOTS
2005 Winner:  Sports Report - SLOTS
2004 Winner:  Anderson/Hester
2003 Winner:  Anderson/Hester
2002 Winner:  CPA Rankings
2001 Winner:  CPA Rankings
2000 Winner:  CPA Rankings






 MOST ACCURATE RETRODICTIVE RATING  -  Complex System 


Winner:  Least Square Regression with team HFA

  The system that had the smallest mean square error this season
was Least Square Regression with team HFA.  This system also
wins this category for the 6th year in a row with a new record average
mean square error of 120.486.  This is 10 points better than the
previous record set in 2006.


2009 Winner:  Least Square Regression with team HFA  (record 120.486)
2008 Winner:  Least Square Regression with team HFA
2007 Winner:  Least Square Regression with team HFA
2006 Winner:  Least Square Regression with team HFA  (record 130.437)
2005 Winner:  Least Square Regression with team HFA
2004 Winner:  Least Square Regression with team HFA
2003 Winner:  The Sports Report - SLOTS
2002 Winner:  CPA Rankings
2001 Winner:  Edward Kambour FOotball Ratings





 MOST ACCURATE RETRODICTIVE RATING  -  Simple System 

Winner:  Least Squares Regression

  It should not be too surprising to see least squares regression do well
here since that is what it does, minimize the square errors.  Least squares
ended with a mean square error of 148.888, anotheer new record.
I am actually very impressed with this basic regression system for college
football.  Predictively it has been near the top of the second half stats every
year and wins this retrodictive category every year.  This system is as simple
as can be and I think you can make a case that it is as good an overall rating
as anything out there.


2009 Winner:  Least Squares Regression  (record 148.888)
2008 Winner:  Least Squares Regression
2007 Winner:  Least Squares Regression
2006 Winner:  Least Squares Regression
2005 Winner:  Least Squares Regression
2004 Winner:  Least Squares Regression
2003 Winner:  Least Squares Regression




BEST RETRODICTIVE SYSTEM -  Complex System 


 Winner:  Least Square Regression with team HFA

 Like the award for best predictive system this award is found by giving each system a
score for each category.  The system that has the highest total is the retroditive
system of the year.

 Overwhelmingly the winner was Least Square Regression with team HFA


2009 Winner:  Least Square Regression with team HFA
2008 Winner:  Least Square Regression with team HFA
2007 WInner:  TSR SLOTS
2006 WInner:  TSR SLOTS
2005 WInner:  TSR SLOTS
2004 Winner:  CPA Rankings
2003 WInner:  TSR SLOTS
2002 Winner:  CPA Rankings
2001 Winner:  CPA Rankings
2000 Winner:  CPA Rankings




BEST RETRODICTIVE SYSTEM  -  Simple system 


Winner: Payne College Football Power Ratings

In this class, the winner was a little know new comer, Payne College Football Power Ratings

2009 Winner:  Payne College Football Power Ratings
2008 Winner:  System median
2007 Winner:  Least Squares
2006 Winner:  Warren Claassen
2006 Winner:  Warren Claassen
2005 Winner:  Peter Wolfe
2004 Winner:  Frank Alder
2003 Winner:  Anderson & Hester