Tuesday, January 20, 2009

The (in)significance of rank in the RPI

In an earlier article, I analyzed the recently proposed changes to the college baseball Ratings Percentage Index (RPI). The RPI's mathematical calculation, which is used to create a number that is calculated to at least four decimal places (e.g. 0.5427), has been well documented. This four decimal number is then used to provide a rank ordering of all teams. Published rankings of the RPI by the NCAA present an interesting but limited view (in that it simultaneously fuels speculation and provides clarity about the worth of one team over another) of how a team will be viewed by an NCAA selection committee.

The ongoing challenge with the RPI in all sports is the perception that it represents a precise measure of one team's quality over another because it is a numeric calculation. This sense of precision is then further reinforced because published RPI listings from the NCAA only provide the rank order of teams, without providing the supporting number that can show incredibly narrow differences (in some cases 1/10,000 of 1 point or less) between teams. The RPI as both a selection tool and information piece for the public could be adjusted in three ways to increase its value.

  1. All versions (public and private) of the RPI should provide the actual calculated number (to four decimal places) for each team so that the magnitude of difference between two teams can be readily understood;

  2. Calculation of the RPI should determine if the differences between each team are statistically significant; and

  3. When the RPI calculation determines that differences between teams are NOT statistically significant, the subsequent rank ordering of teams should list teams these teams as tied in the rankings.

These minor adjustments would clarify public perceptions about the role that RPI plays in at-large selection process for the NCAA Championships. While the RPI is only one tool available for selection, it is a tool that is used repeatedly as a standard measure across which to make comparisons of teams and conferences. These changes would also improve the accuracy of the RPI while providing an on-going reminder that the the RPI is an imperfect tool for selection committees going through their most important duty - determining worthiness of a team to compete for a national championship.


1 comment:

Moonage said...

The problem I have, as you're aware already, is how to do with the problem of pre-season rankings. These initial rankings skew the RPI for the entire season and are totally at the mercy of those making the ranking. This year we saw an incredible bias towards the Big East and ACC and one against the SEC ( yeah, they're down this year, but not THAT down I don't think. ) If NC say, is the #1 pre-season, then everyone playing them gets a boost whether NC is actually any good or not ( see Georgetown pre-season and post-season ). If they played someone like VMI, who was a dangerous as heck team at times this year, they got penalized. In order for the RPI to be a totally believable tool, the pre-season ranking system needs to be taken a lot more seriously. ( And thanks for dropping by my blog! )