Welcome to the 15th annual NCAA PT Awards.
The purpose of these awards is to help raise the publicity for those systems that have been
superior in various qualities of interest.  Most awards will be based entirely on the numbers
I've monitored with my prediction tracker web pages. For the 2015 season I followed the
weekly performance of 68 computer rating  systems. This is down 4 from last year.


  MOST STRAIGHT UP WINNERS (Entire Season)

Winner:  Thompson Average
Winner:  Pi-Rate Mean


straight up plot

  This year I collected the predictions of 781 games involving two FBS teams.
This year I am awarding both Ted Thompson's Average and Pi-Rate Mean as winners.  Thompson had
one more win but Pi-Rate had a missing game in which everyone got it correct. Thompson was
612-169 (78.361%) and Pi-Rate Mean was 611-169 (78.333%).  Thompson's number sets an new
all time record that he previously set in 2013.  Pi-Rate Mean, Thompson ATS, Dokter Entropy
and the midweek line also better the previous record of 78.095%.  So I guess you might say
it was an easy year for picking winners.  A new record has been set in 3 of the last 4 seasons.
I do only count games where both teams are FBS teams, but my guess is this is evidence that
teams are making schedules as weak as they can.
This season the updated line only outperformed the opening line by two games, while the midweek
line did better than both of them.
 In the plots the category winner will be denoted with the dark black line. You can enlarge the plots
by right clicking and selecting view image.  This year followed a little different pattern than last year.
Last year they started low, improved through non conference play then dropped through the end of the season.
This year the numbers started high and pretty much slowly went down the rest of the season.


2015 Winner: Ted Thompson Average record(78.361%), Pi-Rate Mean
2014 Winner: Ted Thompson CAL
2013 Winner: Ted Thompson SPRS  (record 78.095%)
2012 Winner: Hank Trexler
2011 Winner: Covers.com, Tempo Free Gridiron
2010 Winner: Dokter Entropy
2009 Winner: Pigskin Index
2008 Winner: System Average, System Median
2007 Winner: StatFox
2006 Winner: Thompson SPRS, system average, Sagarin predictive, & Sagarin overall
2005 Winner: Pigskin Index
2004 Winner: Jeff Sagarin's Overall Rating
2003 Winner: Edward Kambour & Jeff Sagarin's Overall Rating
2002 Winner: ARGH Power Ratings
2001 Winner: Massey Ratings
2000 Winner: ARGH Power Ratings
1999 Winner: Vegas line



  SMALLEST DEVIATION FROM ACTUAL GAME SCORES (Entire Season)

Winner:  Updated Line

absolute deviation

  Deviation from the game score is the difference between the game prediction
and the actual result.  A value of zero would mean the score difference is predicted
exactly.  One property of a good system would be to minimize the system's average
game deviation.
  The updated line had the smallest average absolute error this year, 12.402 points.
This was very similar to last year when the line lead with a value of 12.477. The
Computer Adjusted line was in second only 0.007 points behind. The numbers had worsened
for three consecutive years, so maybe we have hit a plateau.  As is typical in this
category all of the best systems are systems that are either the line itself or a
derivative of the line.
  From the plot you can see that the line was pretty much the leader for the entire season.
In general the errors were getting larger through the first half of the season and then
leveled off over the second half.


2015 Winner: Updated Vegas Line
2014 Winner: Updated Vegas Line
2013 Winner: Computer Adjusted Line
2012 Winner: Computer Adjusted Line
2011 Winner: Computer Adjusted Line
2010 Winner: Computer Adjusted Line
2009 Winner: Computer Adjusted Line (record 11.26)
2008 Winner: Vegas Line
2007 Winner: Vegas Line
2006 Winner: Computer Adjusted Line
2005 Winner: Vegas Line
2004 Winner: Vegas Line
2003 Winner: Vegas Line
2002 Winner: Vegas Line
2001 Winner: Vegas Line
2000 Winner: Vegas Line
1999 Winner: Vegas Line




  BEST AGAINST THE SPREAD (Entire Season)


Winner:  Updated Line
Winner:  Computer Adjusted Line
Winner:  The Sports Cruncher

ats

  Beating the spread is not the number one goal for most computer
rating systems.  But it is something that the average person likes
to look at and probably the most popular subject that I get emails
about.  I know that looking at all the games is probably not the
most meaningful of measurements but it is what I have been reporting.
The line of record is taken midweek.  Usually Wednesday afternoon or
Thursday morning.
  I really didn't know what to do with this category this year.  The
updated line actually had the highest percent, 269-210, 56.159%.  But what
does that mean?  How can you use the updated line to bet against the midweek
line. By definition the updated line is the current line at the time.
The second contender was the Computer Adjusted Line, 286-234, 55.0%.
The problem with this one is that it is equivalent to the line about a third
of the time by design.  So it may not be a fair comparison since it doesn't
risk the close calls.  Then third was The Sports Cruncher, 412-347, 54.282%
But there is a problem with this one as well because his numbers are one of the
only ones that typically don't come out until after the midweek line.  So
perhaps that is not fair either.  So the end result is that I am mentioning them
all.  Overall the numbers were average. Averaging our right around 50% which
is what I believe should happen. Basically every system was somewhere between
46 and 54%.
  It is interesting the line lead this category for the majority of the season.
The pattern we saw all season long was the midweek line would move off the
opening line and then eventually get pushed back toward the opening value. The
result being the updated line was beating the midweek line.


2015 Winner: Updated Line, Computer Adjusted Line, The Sports Cruncher
2014 Winner: Laffaye RWP
2013 Winner: CPA Rankings (record 56.906%)
2012 Winner: Laz Index, Computer Adjusted Line
2011 Winner: Dunkel Index
2010 Winner: Catherwood Ratings
2009 Winner: Covers.com
2008 Winner: Stephen Kerns
2007 Winner: Nutshell Sports
2006 Winner: Harmon Forecast  (record 56.4% against updated line)
2005 Winner: Keeper's College Football Ratings
2004 Winner: Edward Kambour
2003 Winner: Edward Kambour
2002 Winner: Dunkel Index
2001 Winner: BMC Picks
2000 Winner: Dunkel Index
1999 Winner: Average across all systems


  SMALLEST AVERAGE GAME BIAS (Entire Season)

Winner:  SportsTrends

Bias

  Bias is a little different from deviation.  Deviation measures the
distance between a prediction and the actual result.  Bias combines
distance and location of the prediction.   Bias measures whether the
predictions are too high or too low.  It could be specified in different
terms but my numbers are relative to the home team.
 SportsTrends had the lowest average bias at -0.0164.  As you can see in the
plot the number finished mostly positive, or giving too many points to the home
team. But you can see the season started off with big underestimates of the home
team in week one.  Can you say powder puff schedules.  This was exactly the same
pattern as last year.  By the end of the year we ended up giving too many points
to the home team on average.


2015 Winner: SportsTrends
2014 Winner: Billingsley Report
2013 Winner: Ted Thompson Average
2012 Winner: College Football By the Numbers
2011 Winner: The Pi-Rate Ratings
2010 Winner: Sagarin Predictor (record 0.0010)
2009 Winner: CPA Retro Rankings
2008 Winner; Opening Line
2007 Winner: Thompson SPRS
2006 Winner: Congrove Computer Rankings
2005 Winner: Computer Adjusted Line
2004 Winner: CPA Rankings
2003 Winner: Edward Kambour
2002 Winner: Gupta Power Ratings
2001 Winner: Flyman
2000 Winner: ARGH Power Ratings
1999 Winner: Jeff Sagarin




  MOST ACCURATE PREDICTOR (Entire Season)

Winner:  Updated Vegas Line.

Bias

  This award is based on mean square error. Mean square error takes
into account both deviation and bias and is perhaps the most commonly
used measure of evaluating estimators.
  Not really much to say here, the results are very similar to
absolute error. The updated line had the lowest value at 250.418


2015 Winner: Updated vegas line
2014 Winner: Updated vegas line
2013 Winner: Computer Adjusted Line
2012 Winner: Computer Adjusted Line
2011 Winner: Computer Adjusted Line
2010 Winner: Computer Adjusted Line
2009 Winner: The Vegas Line  (record 206.152)
2008 Winner: The Vegas Line
2007 Winner: The Vegas Line
2006 Winner: The Vegas Line
2005 Winner: The Vegas Line
2004 Winner: The Vegas Line
2003 Winner: The Vegas Line
2002 Winner: The Vegas Line
2001 Winner: The Vegas Line
2000 Winner: The Vegas Line
1999 Winner: The Vegas Line



    ------------------------



  It is debatable how meaningful rating systems are in the early
parts of the season.  How can you rate the teams in the first
week of the season when no teams have played a game yet?  The playoff
committee waits until mid October before releasing its first ranking
of teams.  Some systems choose to wait until around this time
before being made public. Looking at only the second half of the
season also gives an estimate of how well a system does based
only(or mostly) on data from the current year.


   MOST STRAIGHT UP WINNERS (Second Half of Season)

Winner:  Thompson Average

Bias


  The second half data started with week 8 and consisted of 421 games
between two FBS teams. Thompson Average had the best record 327-91, 78.23%.
This sets a new record for the category. Sagarin points finished one game behind.
Both of these bettered the old record of 77.688% set by Massey in 2009.
Like the full season, the second half appears to have been easy to predict.
The winning record from last season would have only placed 37th this season.


2015 Winner:  Thompson Average   (record 78.23%)
2014 Winner:  Dave Congrove
2013 Winner:  CPA Rankings
2012 Winner:  PerformanZ Ratings
2011 Winner:  Tempo Free Gridiron.
2010 Winner:  Brent Craig
2009 Winner:  Massey Ratings
2008 Winner:  Nutshell Sports
2007 Winner:  Edward Kambour
2006 Winner:  Ed Bemiss
2005 Winner:  Dunkel Index
2004 Winner:  Jeff Sagain, Wesley Colley
2003 Winner:  Born Power Index & Edward Kambour
2002 Winner:  Born Power Index
2001 Winner:  Chris Montgomery
2000 Winner:  Geoff Freeze




  SMALLEST DEVIATION FROM ACTUAL GAME SCORES  (Second Half of Season)

Winner:  Updated Line

Bias

The usual suspects are all at the top again here.  The best for the second half was
the updated line with an average error of 12.4879.  Again I ask you to think about the
meaning of that.  The best computer prediction of the final margin was the line and yet it
was off by an average of about 12.5 points per game.


2015 Winner:  Updated Line
2014 Winner:  Ashby Accuratings
2013 Winner:  CPA Rankings
2012 Winner:  Computer Adjusted Line
2011 Winner:  Vegas Line
2010 Winner:  Computer Adjusted Line
2009 Winner:  Computer Adjusted Line (record 11.2917)
2008 Winner:  StatFox
2007 Winner:  Edward Kambour
2006 Winner:  Computer Adjusted Line
2005 Winner:  Vegas Line
2004 Winner:  Ken Ashby AccuRatings
2003 Winner:  Vegas Line (opening)
2002 Winner:  Vegas Line
2001 Winner:  Vegas Line
2000 Winner:  Vegas Line





  SMALLEST AVERAGE GAME BIAS  (Second Half of Season)

Winner:  CPA Retro

Bias


  CPA Retro had the lowest bias in the second half, -0.0599.  The majority of
the systems were on the positive side, the median being about +0.75 points. The
numbers started off very low for a few weeks before turning overwhelmingly positive.
CPA retro was one of only a few to finish below zero.

2015 Winner:  CPA Retro
2014 Winner:  Laz Index
2013 Winner:  Colley matrix
2012 Winner:  College Football by the Numbers
2011 Winner:  Stat Fox
2010 Winner:  Sonny Moore
2009 Winner:  Nutshell Girl
2008 Winner:  Dunkel Index
2007 Winner:  CPA Rankings
2006 Winner:  Vegas Line
2005 Winner:  Brent Moore
2004 Winner:  Least Squares Regression
2003 Winner:  Darrly Marsee
2002 Winner:  Massey BCS
2001 Winner:  PerformanZ Ratings
2000 Winner:  Darryl Marsee's Rankings




  BEST AGAINST THE SPREAD  (Second Half of Season)


Winner:  Updated Line
Winner:  Computer Adjusted Line
Winner:  The Sports Cruncher


Bias

  Like the full season stats there is reason to spread the this around. The updated line and
Computer adjusted lines for the same reasons as for the full season.  SportsTrends, like
the computer adjusted line does not make picks on every game. The line had the best record
at 130-101, 56.277%.  Sports trends was second at 195-153, 55.908. And the computer adjusted
line was next at 141-115, 55.078%  Pi-Rate Bias and Atomic Football were the only other
systems to do better then 53% ATS in the second half.  Interestingly the computer average was
3rd to last at only 44.01%  The majority of the systems finished below 50% this year.  So the
plot looks very striking with the two lines performing so much better than the rest for the
duration of the second half.


2015 Winner:  Updated Line, Computer Adjusted Line, SportsTrends.
2014 Winner:  Massey Consensus Rank
2013 Winner:  CPA Rankings (Record 59.322%)
2012 Winner:  Least Squares with team HFA
2011 Winner:  Dunkel Index
2010 Winner:  Least squares regression
2009 Winner:  Sonny Moore
2008 Winner:  StatFox
2007 Winner:  Nutshell Sports
2006 Winner:  Ed Bemiss
2005 Winner:  The Sports Report
2004 Winner:  StatFox          (record 58.94% for updated line)
2003 Winner:  Edward Kambour
2002 Winner:  The Sports Report
2001 Winner:  BMC Picks
2000 Winner:  The Buck System



  MOST ACCURATE PREDICTOR (Second Half of Season)

Winner: Midweek Line

Bias


  The midweek line had the smallest average mean square error over the second
half, 254.896.  Followed by the updated line, computer adjusted line, and
opening line.  I think you can get the idea.  The line is the best predictor
of actual game margin.  Regardless if it is from the end of the week, middle
of the week, or even the start of the week.


2015 Winner:  Midweek Line
2014 Winner:  Thompson CAL
2013 Winner:  CPA Rankings
2012 Winner:  Computer Adjusted Line
2011 Winner:  Computer Adjusted Line
2010 Winner:  Computer Adjusted Line
2009 Winner:  Computer Adjusted Line    (record 208.618)
2008 Winner:  Born Power Index
2007 Winner:  Atomic Football
2006 Winner:  Vegas Line
2005 Winner:  Vegas Line
2004 Winner:  Ashby Accuratings
2003 Winner:  Vegas Line
2002 Winner:  Vegas Line
2001 Winner:  Vegas Line
2000 Winner:  Vegas Line









  BEST PREDICTIVE SYSTEM  

Winner:  Atomic Football

  To come up with the best overall predictive system I give each system
points for how well they do in all of the above categories.  I then sum up
the points and the system with the highest total is named predictive system
of the year.  So this award goes to the system that is the most well rounded
One flaw can be enough to pull a system out of the running.
  This award often generates a surprise winner in seasons like this past year
where no one system dominates across the board.  This year that system is
James Ashburn's Atomic Football. Well technically, Atomic Football came in
third place in my rankings.  The top three were the same order for both the
full season and the second half.  They were the updated line in first place,
the computer adjusted line in second place and Atomic Football in third
place.  I prefer not giving this award to the line, otherwise it would win
too often.  So as long as there is another solid option they will get the nod.
This year Atomic Football was 10th straight up, 7th ATS, and 14th in error.


2015 Winner:  Atomic Football
2014 Winner:  Billingsley+
2013 Winner:  CPA Rankings
2012 Winner:  System Average
2011 Winner:  Dunkel Index
2010 Winner:  Catherwood Ratings
2009 Winner:  Ed Kambour
2008 Winenr:  StatFox
2007 Winner:  Harmon Forecast
2006 Winner:  Computer Adjusted Line
2005 Winner:  Dunkel Index
2004 Winner:  Ken Ashby AccuRatings
2003 Winner:  Ed Kambour
2002 Winner:  Ed Kambour
2001 Winner:  Kenneth Massey (non-BCS)
2000 Winner:  ARGH Power Ratings