OthelloRank - A First Attempt at a Non-biased BCS Computer Ranking Algorithm
I've been frustrated over the complete arbitrariness of deciding which of Michigan, USC and Florida is most worthy of a matchup against Ohio State in the national championship. I wanted to see for myself if I could come up with completely objective and non-arbitrary ranking algorithm. I was inspired by the simplicity of the game Reversi/Othello. A more thorough description of the algorithm used follows.
When I finally got around to running the program as I had initially designed in my head, I ended up with Ohio State ranked #1, Boise State ranked #2, Michigan #3. I realized that I had a few parameters I could tweak ( i.e. how many levels of propagation should be counted, how many points to assign a loss, or a secondary loss, etc. Eventually, I concluded that it's impossible to make these decisions without introducing my own bias into the results, thus negating my purpose for creating the entire experiment. I guess I'd have to start by configuring the rankings against historical results without taking the current season into account at all. Once I've fine-tuned things, I can turn around and use the algorithm with confidence that I haven't introduced my own bias. If they don't scrap the BCS entirely, I might try to work on that for next season.
I did come up with some pretty convincing results with about five hours of work, half of which was writing a program to extract the teams, schedules and results from ESPN's website. In the end, I decided to only propagate out two levels, and to count wins as 1 point, and those opponent's wins as a second point, losses as -15 points, and those opponent's losses as -10 points. Using this approach, I have my top four as #1. Ohio State, #2 Florida, #3 USC, #4 Michigan.
In the end, I ended up solving nothing. I can make my rankings look however I want, depending on how much emphasis I put on losses. It is readily apparent that both Florida and USC had tougher schedules than Michigan. If Michigan had chosen a tougher SEC or PAC10 opponent instead of Ball State, Vanberbilt, or Central Michigan as another out-of-conference opponent, they would've had enough to overtake USC/Florida, but as it stands, it's hard to make a case. Florida gets the nod over USC because a loss against Auburn doesn't cost as much as a loss against Oregon State.
I guess when it comes down to it, this all points to the necessity of a playoff system. On the one hand, a playoff system may somewhat diminish the importance of the regular system, but on the other, it's incredibly agonizing to see Michigan come so close to a national championship, but fall short the series of belts, pulleys, gears and kludges known as the BCS.
Download Links:
OthelloRank Executable: http://www.filefactory.com/file/80f2f3/ (Note: .NET 2.0 Framework is required - 37 KB)
OthelloRank Source Code: http://www.filefactory.com/file/9c12f6/ (Written using VS2005/C# - 77 KB)
Results
Top Twelve
1. Ohio State 20.25
2. Florida 19.27
3. USC 19.00
4. Michigan 18.92
5. Louisville 16.45
6. Boise State 13.27
7. Wisconsin 12.82
8. Rutgers 12.00
9. LSU 11.92
10. Notre Dame 11.17
11. Arkansas 9.55
12. Auburn 9.42
********************************
Michigan Team Report
********************************
Wins Against (11 total)
------------------------
Vanderbilt (3-8) SCORE: 12
Central Michigan (8-4) SCORE: 18
Notre Dame (10-2) SCORE: 42
Wisconsin (10-1) SCORE: 26
Minnesota (5-6) SCORE: 22
Michigan State (4-8) SCORE: 17
Penn State (7-4) SCORE: 21
Iowa (5-6) SCORE: 22
Northwestern (4-7) SCORE: 14
Ball State (5-6) SCORE: 12
Indiana (5-6) SCORE: 22
Losses Against (1 total)
------------------------
Ohio State (12-0) SCORE: -1
Cumulative score is 227
Weighted Score is 18.92
--------------------------------------
********************************
USC Team Report
********************************
Wins Against (10 total)
------------------------
Arkansas (9-2) SCORE: 31
Nebraska (8-3) SCORE: 28
Arizona (5-6) SCORE: 26
Washington State (6-6) SCORE: 21
Washington (5-7) SCORE: 22
Arizona State (6-5) SCORE: 25
Stanford (1-10) SCORE: 8
Oregon (6-5) SCORE: 27
California (7-3) SCORE: 25
Notre Dame (10-2) SCORE: 42
Losses Against (1 total)
------------------------
Oregon State (7-4) SCORE: -46
Cumulative score is 209
Weighted Score is 19
********************************
Florida Team Report
********************************
Wins Against (10 total)
------------------------
Southern Miss (7-4) SCORE: 20
UCF (3-8) SCORE: 9
Tennessee (9-3) SCORE: 37
Kentucky (6-5) SCORE: 24
Alabama (6-6) SCORE: 20
LSU (10-2) SCORE: 40
Georgia (7-4) SCORE: 31
Vanderbilt (3-8) SCORE: 12
South Carolina (6-5) SCORE: 24
Florida State (6-6) SCORE: 31
Losses Against (1 total)
------------------------
Auburn (10-2) SCORE: -36
Cumulative score is 212
Weighted Score is 19.27
************************************
About OthelloRank
************************************
OthelloRank is an attempt to build a completely objective BCS computer ranking algorithm. There is no preseason rankings, no assumptions, no margins of victory, or anything else of that sort. The only item that is looked at is wins and losses.
The algorithm is called OthelloRank, and it loosely based on the basic concepts of the board game Othello (Also known as Reversi). In Othello, the object of the game is to capture as many pieces as possible, thereby changing the polarity of those pieces on the game board that don't already belong to you.
Assume that a win represents a white game piece, and a loss represents a black game piece. In Othello, the simplest move you can perform is to surround and capture one opposing piece on the game board. In OthelloRank, each game played is considered one move. The OthelloRank equivalent to capturing a single game piece is to defeat an opponent that has zero wins. This move flips the polarity of a single team, and provides you with one point.
Moving back to the board game, With a strategically placed move, it is possible to capture pieces in several different directions with a single move. This would be considered a quality move. The equivalent in OthelloRank is a "Quality Win". If you defeat an opponent with five wins on the season. The polarity of each of those teams switches, giving you six points. In addition, additional points can be won by proxy. Each sub-opponent that one of those opponents has defeated can also have their polarity switched, resulting in additional points. This process propagates on outward until either the configured limit, or until all possible outcomes have been considered. Each opponent's polarity can only be switched once, resulting in a maximum theoretical score of 118 points per turn (You cannot obtain points)
Consider the reverse - A loss in OthelloRank is the equivalent of an opponent taking a turn in the game of Othello. Polarities are flipped the opposite direction, resulting in a net loss of points. Losses are propagated by proxy, meaning that if you lose against a strong team, with no losses themselves, you will only lose one point (The exact number of points is configurable), but if you lose against a weak team, with many losses, many additional points may be lost.
Each game played by a team is considered. Games played against non-Division I-A opponents are thrown out entirely. The final rank is generated by dividing the cumulative score achieved by the # of games played. The maximum theoretical score is 118, and the minimum theoretical score is -118.
At a glance, this method may appear to give teams that play in weaker conferences a competitive advantage, since they have inflated records by playing against weaker competition. The theory is, that, over the course of a season, the opponents in those weaker conferences will have compiled worse non-conference schedules, resulting in fewer available net points available. The goal with this ranking algorithm is to eliminate all arbitrary assumptions, and allow the computer to decide which team is the strongest.
When I finally got around to running the program as I had initially designed in my head, I ended up with Ohio State ranked #1, Boise State ranked #2, Michigan #3. I realized that I had a few parameters I could tweak ( i.e. how many levels of propagation should be counted, how many points to assign a loss, or a secondary loss, etc. Eventually, I concluded that it's impossible to make these decisions without introducing my own bias into the results, thus negating my purpose for creating the entire experiment. I guess I'd have to start by configuring the rankings against historical results without taking the current season into account at all. Once I've fine-tuned things, I can turn around and use the algorithm with confidence that I haven't introduced my own bias. If they don't scrap the BCS entirely, I might try to work on that for next season.
I did come up with some pretty convincing results with about five hours of work, half of which was writing a program to extract the teams, schedules and results from ESPN's website. In the end, I decided to only propagate out two levels, and to count wins as 1 point, and those opponent's wins as a second point, losses as -15 points, and those opponent's losses as -10 points. Using this approach, I have my top four as #1. Ohio State, #2 Florida, #3 USC, #4 Michigan.
In the end, I ended up solving nothing. I can make my rankings look however I want, depending on how much emphasis I put on losses. It is readily apparent that both Florida and USC had tougher schedules than Michigan. If Michigan had chosen a tougher SEC or PAC10 opponent instead of Ball State, Vanberbilt, or Central Michigan as another out-of-conference opponent, they would've had enough to overtake USC/Florida, but as it stands, it's hard to make a case. Florida gets the nod over USC because a loss against Auburn doesn't cost as much as a loss against Oregon State.
I guess when it comes down to it, this all points to the necessity of a playoff system. On the one hand, a playoff system may somewhat diminish the importance of the regular system, but on the other, it's incredibly agonizing to see Michigan come so close to a national championship, but fall short the series of belts, pulleys, gears and kludges known as the BCS.
Download Links:
OthelloRank Executable: http://www.filefactory.com
OthelloRank Source Code: http://www.filefactory.com
Results
Top Twelve
1. Ohio State 20.25
2. Florida 19.27
3. USC 19.00
4. Michigan 18.92
5. Louisville 16.45
6. Boise State 13.27
7. Wisconsin 12.82
8. Rutgers 12.00
9. LSU 11.92
10. Notre Dame 11.17
11. Arkansas 9.55
12. Auburn 9.42
******************************
Michigan Team Report
******************************
Wins Against (11 total)
------------------------
Vanderbilt (3-8) SCORE: 12
Central Michigan (8-4) SCORE: 18
Notre Dame (10-2) SCORE: 42
Wisconsin (10-1) SCORE: 26
Minnesota (5-6) SCORE: 22
Michigan State (4-8) SCORE: 17
Penn State (7-4) SCORE: 21
Iowa (5-6) SCORE: 22
Northwestern (4-7) SCORE: 14
Ball State (5-6) SCORE: 12
Indiana (5-6) SCORE: 22
Losses Against (1 total)
------------------------
Ohio State (12-0) SCORE: -1
Cumulative score is 227
Weighted Score is 18.92
------------------------------
******************************
USC Team Report
******************************
Wins Against (10 total)
------------------------
Arkansas (9-2) SCORE: 31
Nebraska (8-3) SCORE: 28
Arizona (5-6) SCORE: 26
Washington State (6-6) SCORE: 21
Washington (5-7) SCORE: 22
Arizona State (6-5) SCORE: 25
Stanford (1-10) SCORE: 8
Oregon (6-5) SCORE: 27
California (7-3) SCORE: 25
Notre Dame (10-2) SCORE: 42
Losses Against (1 total)
------------------------
Oregon State (7-4) SCORE: -46
Cumulative score is 209
Weighted Score is 19
******************************
Florida Team Report
******************************
Wins Against (10 total)
------------------------
Southern Miss (7-4) SCORE: 20
UCF (3-8) SCORE: 9
Tennessee (9-3) SCORE: 37
Kentucky (6-5) SCORE: 24
Alabama (6-6) SCORE: 20
LSU (10-2) SCORE: 40
Georgia (7-4) SCORE: 31
Vanderbilt (3-8) SCORE: 12
South Carolina (6-5) SCORE: 24
Florida State (6-6) SCORE: 31
Losses Against (1 total)
------------------------
Auburn (10-2) SCORE: -36
Cumulative score is 212
Weighted Score is 19.27
************************************
About OthelloRank
************************************
OthelloRank is an attempt to build a completely objective BCS computer ranking algorithm. There is no preseason rankings, no assumptions, no margins of victory, or anything else of that sort. The only item that is looked at is wins and losses.
The algorithm is called OthelloRank, and it loosely based on the basic concepts of the board game Othello (Also known as Reversi). In Othello, the object of the game is to capture as many pieces as possible, thereby changing the polarity of those pieces on the game board that don't already belong to you.
Assume that a win represents a white game piece, and a loss represents a black game piece. In Othello, the simplest move you can perform is to surround and capture one opposing piece on the game board. In OthelloRank, each game played is considered one move. The OthelloRank equivalent to capturing a single game piece is to defeat an opponent that has zero wins. This move flips the polarity of a single team, and provides you with one point.
Moving back to the board game, With a strategically placed move, it is possible to capture pieces in several different directions with a single move. This would be considered a quality move. The equivalent in OthelloRank is a "Quality Win". If you defeat an opponent with five wins on the season. The polarity of each of those teams switches, giving you six points. In addition, additional points can be won by proxy. Each sub-opponent that one of those opponents has defeated can also have their polarity switched, resulting in additional points. This process propagates on outward until either the configured limit, or until all possible outcomes have been considered. Each opponent's polarity can only be switched once, resulting in a maximum theoretical score of 118 points per turn (You cannot obtain points)
Consider the reverse - A loss in OthelloRank is the equivalent of an opponent taking a turn in the game of Othello. Polarities are flipped the opposite direction, resulting in a net loss of points. Losses are propagated by proxy, meaning that if you lose against a strong team, with no losses themselves, you will only lose one point (The exact number of points is configurable), but if you lose against a weak team, with many losses, many additional points may be lost.
Each game played by a team is considered. Games played against non-Division I-A opponents are thrown out entirely. The final rank is generated by dividing the cumulative score achieved by the # of games played. The maximum theoretical score is 118, and the minimum theoretical score is -118.
At a glance, this method may appear to give teams that play in weaker conferences a competitive advantage, since they have inflated records by playing against weaker competition. The theory is, that, over the course of a season, the opponents in those weaker conferences will have compiled worse non-conference schedules, resulting in fewer available net points available. The goal with this ranking algorithm is to eliminate all arbitrary assumptions, and allow the computer to decide which team is the strongest.
posted by markh100 at
1:32 PM
