Tuesday, January 27, 2009

Scheduling to improve your Ratings Percentage Index (RPI) - The Prisoner's Dilemma

Nearly every week I find myself in conversations about how to schedule in a manner that benefits the Ratings Percentage Index. It comes up because the RPI is woven throughout the selection process for at-large bids to the NCAA tournament. Everyone wants to know, how do I improve my team's RPI and how does our conference get more bids to the NCAA Tournament? The easy answer is to win every game against the toughest schedule in the country. But as I wrote in an earlier post regarding college baseball, winning games isn't enough. It isn't valued highly enough in the current formula. What matters most is your opponent's record. It is this reality that suggests The Prisoner's Dilemma, a classic problem in game theory, may provide guidance. The following is an excellent explanation (in italics) from Wikipedia.

In its classical form, the prisoner's dilemma (PD) is presented as follows: Two suspects are arrested by the police. The police have insufficient evidence for a conviction, and, having separated both prisoners, visit each of them to offer the same deal. If one testifies (defects) for the prosecution against the other and the other remains silent, the betrayer goes free and the silent accomplice receives the full 10-year sentence. If both remain silent, both prisoners are sentenced to only six months in jail for a minor charge. If each betrays the other, each receives a five-year sentence. Each prisoner must choose to betray the other or to remain silent. Each one is assured that the other would not know about the betrayal before the end of the investigation. How should the prisoners act? (The graphic to the left illustrates the problem, using 20 years and 1 year instead of 10 years and 6 months but the premise remains unchanged.)

If we assume that each player prefers shorter sentences to longer ones, and that each gets no utility out of lowering the other player's sentence, and that there are no reputation effects from a player's decision, then the prisoner's dilemma forms a non-zero-sum game in which two players may each cooperate with or defect from (i.e., betray) the other player. In this game, as in all game theory, the only concern of each individual player (prisoner) is maximizing his/her own payoff, without any concern for the other player's payoff. The unique equilibrium for this game is a Pareto-suboptimal solution—that is, rational choice leads the two players to both play defect even though each player's individual reward would be greater if they both played cooperatively.

In the classic form of this game, cooperating is
strictly dominated by defecting, so that the only possible equilibrium for the game is for all players to defect. No matter what the other player does, one player will always gain a greater payoff by playing defect. Since in any situation playing defect is more beneficial than cooperating, all rational players will play defect, all things being equal.

So what does the prisoner's dilemma have to do with scheduling and the RPI?
Just as in the prisoner's dilemma where criminals are seeking an optimal outcome, teams are facing a "scheduling dilemma" - that is, trying to determine the best choice to achieve an optimal outcome, in this case an NCAA bid and secondarily a higher seed in the tournament.
In the prisoner's dilemma, there are two choices (defect or silent). Likewise in the scheduling dilemma there are 2 potential "choices", winning or losing an individual contest (and enhancing the possibility of winning, a function of the strength of an opponent).

In the Prisoner's Dilemma the two choices can produce four potential outcomes for each criminal - no jail time, six months, 5 years, or 10 years in jail. In the scheduling dilemma the two choices can only result in two potential outcomes - a tournament bid via at-large or an AQ selection or no tournament bid.

The dilemma for the scheduler is whether to schedule difficult teams to potentially enhance your RPI while simultaneously increasing the possibility of losing OR scheduling weak teams which will hurt your RPI but simultaneously increase your winning percentage. Scheduling independently, just as in the prisoner's dilemma where the prisoner's do not speak to one another, teams will opt for the first choice - scheduling strong opponents - because it appears to be in their best interest and because they are unsure of what the actions of their conference partners will be.

But unlike the prisoner's dilemma, teams are not constrained by separation and can freely discuss their non-conference scheduling intentions. It is this key difference that leads me to believe that communication between conference members can (and in some cases may already) effect conference RPI rankings by having all conference members take steps to insure high winning percentages. Some of this is due to the quality of the conference's teams (because some teams are of course substantially better than others and have superior resources) but some is due to scheduling in a manner that is conducive to high winning percentages (e.g. consistently/exclusively scheduling weaker teams, playing the vast majority of games at home, etc.). Obviously winning games is important for enhancing the RPI, but having your conference members achieve high winning percentages is more valuable than winning yourself because the most important factor in the RPI is who you play. This factor is then compounded by the opponent's opponent's winning percentage - the majority of whom are your conference peers - because of the multiple contests against one another far exceed any other set of common opponents.

While it is true that all the schedule planning you undertake is irrelevant if you don't win games, if the goal is to optimize the winning percentage of your opponents, having all conference members agree to compete against lesser opponents helps this occur.

The math seems to suggest that cooperative (perhaps even collusive) scheduling could enhance a team's RPI. While this doesn't guarantee selection into the NCAA tournament, the enhancement of a team's RPI (and just as importantly their conference peers' RPI) through this method begins to make selection more likely for many reasons including:

  • Numbers are concrete and easily understood - if asked to defend the selection or non-selection of a team, its difficult to argue with a formula based rank; and

  • The vast majority of comparisons that occur during NCAA selections are based on the RPI (e.g. record vs the Top 25 in the RPI, record vs the the top 50, the top 100, strength of schedule, non-conference strength of schedule, etc.)

These comparisons are particularly crucial when the last at-large spots for an NCAA tournament are being selected. Typically, two types of teams compete for the last at-large spots: a) additional representatives from a conferences that already have numerous bids; or b) teams from conferences trying to get a second bid in addition to their conference automatic qualifier, who are often strong second place teams or first place teams from weaker conferences who are upset in their conference tournament.

Generally teams from "power conferences" get more at-large bids. This can happen for a number of reasons. Ideally, it will be because the team is truly better. Skeptics will argue that it is due to brand name recognition, history, tradition or politics. But another possible reason is that the committee when choosing these final teams constantly compares the criteria cited above along with information such as:

  • an individual team's performance against teams already in the tournament; and

  • an individual team's performance against team's under consideration for selection.

The resulting selection of "power conference teams" reflects a rational choice by selection committees because there are so many more "defensible" points of comparison on which to base a decision, and these points of comparison are often conference opponent comparisons. Weaker teams from stronger conferences compare favorably through a halo effect from their conference partners. Stronger teams from weaker conferences simply have fewer comparison points and are not able to benefit from a similar halo. Indeed the rising tide lifts all boats. So you have to make the tide rise. How?

When it comes to athletic scheduling and the RPI, there are only 2 things you can control, who you play and your team's performance (which effects winning or losing). Thus, when using a formula that weights the winning percentage of your opponent more highly than other factors, creating high winning percentages for the opponents you play most often (other conference members) makes perfect sense. And this is best achieved through collective conference efforts to win - with superior talent and/or beneficial scheduling.

Most teams will continue to struggle with the dilemma's of scheduling because there are so many other factors that influence scheduling decisions in addition to the RPI - the need to generate revenue or cut costs, maintaining traditional rivals, enhancing the student athlete experience and the like. Further, just as in the prisoner's dilemma, you have to trust your partner in crime to do the right thing or else you suffer and they benefit. The result is that most teams will continue to act independently. But there's a reason the same conferences are highly represented in the post season every year - they win games and have top teams, but when the last selections are made, some teams may appear better than they truly are. Creating the appearance of quality through independent scheduling is hard. You need help from your conference "friends" - truly a prisoner's dilemma.


Anonymous said...

Nice article but it just takes the world as a given. Why not remove all the whining and create an all comers tourney - the NCAA "open." This would remove all crossed incentives from scheduling. Schedule whomever your want (weak, strong, in between) and know that in the end, a single elimination format tourney will decide a champion.

Anonymous said...

Although the original post is interesting, I'm not sure part of its premise is correct. It states that strength of schedule is more important, in computing the RPI, than winning record. On the surface of the RPI formula, this seems correct: winning record is weighted at 25%, opponents' winning record at 50%, and opponents' opponents' winning record at 25%. This, however, is deceptive. I have done a detailed study of the RPI as applied to Division I women's soccer. If one studies the data, what one finds is that the spread from the top teams to the bottom teams in their winning percentages is much greater than the spread in teams' opponents' winning percentages (because there are multiple opponents) and much, much greater than the spread in teams' opponents' opponents' winning percentages. When one takes these differences in spreads into account, the effective weight of the three RPI elements is 50% winning record, 40% opponents' winning record, and 10% opponents' opponents' winning record.

In addition, I have done a study to match conferences' average RPIs against the conferences' teams actual performances in games. What I found is that teams from the strongest conferences, as indicated by their average RPIs, actually outperformed their RPIs. In other words, they won more games than their RPIs said they should and the lost less games than their RPIs said they should. On the other hand, the converse was true teams from the weakest conferences, as indicated by their average RPIs. In other words, the RPI actually underrates teams from strong conferences and overrates teams from weak conferences. So, if the RPI says a conference's teams are strong, they actually are even stronger than the RPI says. And if the RPI says a conference's teams are weak, they actually are even weaker than the RPI says. Thus, contrary to what some middle-conference fans have asserted, the RPI actually favors their conferences in relation to the perennially strong conferences.

Michael Cross said...

The difference in women's soccer is that teams only play approximately 20 games, compared to the nearly 60 played by baseball and softball. In fact soccer has one of the smallest sample sizes available.

Also, the RPI, "gets it right" at the top and bottom ends of the spectrum, great teams are always great, no matter the formula. Weak teams are always weak. The problem is at the point where the bubble teams are and selections are made. This year's baseball selections are a great example. Weak temas from strong conferences were rewarded while strong teams from weak conferences were punished.

Brad said...

I have the same problem with the RPI as I do with the BCS (and the reason I'm against a playoff for now). Its the criteria set that is weighing things into the hands of the BCS schools.

For both formulas:

1. Everyone must play 50% of their games at home and 50% on the road. Winning on the road is a premium - teams like Syracuse, Duke, UK hate leaving their respected arenas. Hell Western Kentucky over the past 2 seasons is 26-2 at home...

2. Adjust the formula - 50% Winning / 40% Opp Winning % / 5% Opp - Opp winning % / 5% weighting on # of road and home games played

3. Must be at least 1 game over .500 in your league to qualify for the NCAA Tournament - that should fix itself with the formula adjustment.

4. 64 plays 1 / 63 plays 2 / 62 plays 3 - etc.

***Wanna get really creative? Blind draw the entire tournament. In essence Kentucky could play Duke and UNC could play Kansas in the first round. No seeding.