Cooper's Conference Column

Alex Cooper's Take on NCAA Conference Affairs

College Football Conference Supremacy Using a Cross-Country Scoring Model – An Update

by Leave a comment

It’s here! The AP preseason college football top 25 poll is out. Temperatures may not be cooling a bit yet, but this is another indication that the seasons are a-changing. We are one step closer to college football, and fans and media now have another high-profile season prediction over which to argue. Part of this argument for several years now has been over which conference in the country is the best. This is a very difficult argument to actually win on anything other than hot air. Do you count bowl victories? National championships won? Non-conference records?

Four years ago, I made an argument on this blog for the efficacy of using a cross-country scoring system as a means by which we can evaluate empirically the old, fun, and ever-present debate of fandoms across college football: which conference is truly the best? Now that I am four seasons wiser, I am still convinced that this scoring method remains a valid measurement of collective success. I won’t rehash it all now (feel free to refresh you memory by reading it here), but the basic idea is that the sport of cross country evaluates both individual performance and group performance in a single competition. We can appropriate that for assessing college football conferences.

In college football, it’s easy to see who the best team is by the end of the season (hint: they’re the ones holding a big, shiny trophy that looks like a miniature version of a futuristic skyscraper). The landscape of college football has shifted in the past 30 years to one of a more conference-based system, meaning teams can rarely go independent of a grouping of other schools of a similar status and be successful. Indeed, the last time a team won the national championship as an independent was in 1989 (Miami [FL] just before joining the Big East). National champions are often from conferences. The conference of the national champion is often elevated in status upon the victory, and rightly so; national championships are worthy of praise and are only captured by programs with a considerable strength. Conferences should celebrate their teams’ success on a national stage.

This narrative of equating a team’s national championship with conference supremacy has only become more prevalent as conferences themselves become more politically powerful in the NCAA and important to fanbases. The SEC perpetuated this argument for years during “The Streak” of 2006-2012 when the conference had 7 consecutive national championships by 4 different teams. However, SEC fans in the years since have been both burned and bolstered by this measurement of success: no conference since The Streak has won even back-to-back titles, so the argument of conference supremacy is constantly debated. In each case of victories by Florida State (ACC, 2013), Ohio State (Big Ten, 2014), Alabama (SEC, 2015 & 2017), and Clemson (ACC, 2016 & 2018), fans, media, and related institutions themselves have fed off of the bragging momentum winning the title has given the champion’s conference as a whole. For example, Wake Forest, a team that finished their regular season right at .500 (6-6) in 2018, congratulated division opponent Clemson on their victory:

This tweet may seem good-spirited and harmless, but a critical eye will see that the Demon Deacons’ program is seeking to bolster the Atlantic Coast Conference’s reputation by promoting a national championship win from a member institution; this win will somehow elevate their own program in the Wake (pun intended) of a year of mediocrity. But However, does a rising tide of a team’s national championship really lift all boats in the conference? It certainly does not hurt. Though, this single achievement alone is often misappropriated by fans of other teams in the conference of the national champion as the data point sufficient enough to pronounce the whole conference as superior. This narrative is perpetuated not only at the tailgate but also on twitter and at the television analysts desk. See this tweet from a sports journalist:

Using a highly selective dataset from the years after The Streak, Machota presents data that shows the ACC on top. As superior to the SEC, Big 10, and all other conferences not listed.

This is a simply insufficient measurement for evaluating conferences. Peter Burns recognizes this:

Within those 6 years Machota cited, there were instances where other teams in the conference of the champion were not up to championship snuff or even nationally prominent. I agree with Burns on the head-scratching, but for a different reason: how can one evaluate an entire grouping of teams by just one team? This leaves out 90% of the on-field play for a conference that year out of the equation.

If only there were another system to measure individual and team success simultaneously.

Fortunately, the sport of cross country has already figured this out. The formula is simple: the place you finish in a race is the number of points you score for your team. Each team adds up its 5 fastest runners’ places (4 in the United Kingdom). Lowest team score wins.

Also fortunately, college football is obsessed with ranking teams. ESPN even has a weekly show dedicated to releasing a Top 25 poll. And these rankings look a lot like a cross-country race result: the best team is ranked first, and the best runner finishes first.

Therefore, to evaluate conferences using a more robust measurement than the binary national-championship-or-no-national-championship metric, I propose using a cross-country scoring system where individual football teams are like individual runners, and football conferences are like the cross-country team that scores based on several of its best members.

My first article on this topic was released at this time in 2015 after Ohio State beat Alabama and Oregon consecutively to win the first College Football Playoff. I pushed back on the popular discourse of the Big 10 Conference (B1G) regaining supremacy in college football (see old twitter commentary from NBA stars and fast-food chains alike). Using a cross-country (XC) scoring method, the B1G that year was not first and not even second. This way, Ohio State’s first-place finish that year certainly helped the conference and was not discounted at all; cross country teams know the value of a first-place finish to a team score. However, every conference was evaluated by an evaluation of their teams that was wider in scope than ESPN or Twitter usually allows for.

Since this time, I’ve kept on scoring conferences in this manner using the AP Top 25 poll. Each week I tabulate the scores to see how throughout the season how the conferences stack up.  And now that the AP has released its preseason 2019 poll, it is time to share this updated data to see how the competitiveness of collegiate athletic conferences has evolved.

As with any scoring model, this methodology has its limitations. The ranking of these schools is done not by an objective clock as in a cross-country race but rather a grouping of individuals with opinions and biases. This article uses exclusively the Associated Press’ poll, though you could easily adapt it to use the USA Today Coach’s Poll that works also by acquiring votes from participants. I’ve chosen the AP which uses journalists rather than coaches, but a narrow margin in the AP XC ranking could be flipped if the USA Today XC rankings. These differences in voting highlight the human element of this model. Additionally, this model is ordinarily-based rather than interval-based. That is, one could use this same methodology but add up the points the AP gives each team based on their votes by conference instead of just adding up the rankings as is done here. This could arguably take into account not just the fact that team #7 is considered “better” than team #8 but quantitatively assess just how much better team #7 is than team #8.

I think the current methodology, though derived from running, is still in line with the spirit of college football. The human element of polling and ranking is something that has caused endless debate in the history of the sport and has not receded since the institution of the Bowl Championship Series (BCS, 1998-2013) or the transition to the College Football Playoff (CFP, 2014-Present). The AP poll is consequential in that debate; though the coach’s poll was used in the old BCS formula, the AP still continues to crown its own champion as the NCAA does not itself award a D1 FBS champion. And until the CFP rankings come out deep into the regular season, the AP poll is used as the premier measurement of the landscape of the season. And these polls matter in the sense that a team is ahead of another, not by how much ahead. The focus is always on the ranking, not the point total. An interval calculation for this XC scoring model would also unhelpfully skew the results in the favor of those top few teams with those further down the ballot mattering very little. The argument for using rankings elevates all teams receiving votes as contributors to their conference’s overall success.

One more consideration: this model has proved to work most effectively when using a conference’s top 4 rather than top 5 schools. There are usually about 35-40 teams receiving votes each week in the AP, and with the number of independent teams or Group-of-5 conference members that usually appear somewhere in the rankings, this 5 team mark is hard to consistently maintain. Consider 2017: each Power-5 conference scored each week by using 4 teams, but three of the five conferences had at least 3 weeks of no scoring when using the 5 team metric. Additionally, conference size is not consistent; the Big XII has ten teams and the Pac 12 has twelve. The ACC, SEC, and B1G all have fourteen, making that 5th slot easier to grab. So here, we’ll score like the British in cross country and count 4 teams.

With all this in mind, let’s get to the data. First, we’ll do a deep dive into the past two seasons to see how this methodology works throughout a season. Then, we’ll zoom out to see the totality of conference competitiveness during the post-Streak era in which our earlier tweets were from.

2017: Preseason Expectations Underwhelmed

Coming off a thrilling Clemson (ACC) national championship victory the season before, the AP poll preceding the 2017-18 season showed a close race for the best conference off the bat with one clear front-runner: the Big Ten. The Midwestern conference had finished 2016 on a good run and as the best conference by the AP XC metric, and expectations for the next season were high. B1G schools accounted for 4 of the top 11 slots. And thus, the conference’s overall score was very low to place it initially far ahead its other competitors.

AP XC 2017 Preseason Results

SEC ACC Big 10 Big XII Pac-12
41 42 28 43 50
1 3 2 6 4
12 5 6 7 8
13 16 9 10 14
15 18 11 20 24
17 21 30 22 27
25 29 39 23 32
41.5 31 43 26 34
41.5 35.5

But as the season progressed, the landscape began to change. Through the first seven or eight weeks, the look of the B1G out front with a pack in the middle with the other four conferences was becoming normal. Then Weeks 9 and 10 changed everything for the B1G. In Week 9, two of the conference’s ranked schools lost, including #2 Penn State who lost a big lead against Ohio State. The same thing happened again in Week 10, this time most notably to Ohio State against an unranked Iowa team. Meanwhile, 4 ranked SEC teams won in Week 10, and the Pac 12 had several ranked losses. This flipped the narrative of the season on its head. The Big XII, ACC, and Pac 12 eventually finished the regular season well above the B1G and SEC, and an abysmal 1-8 bowl record for the Pac 12 placed them last at the end of the season. The SEC maintained its position from Week 11 on, rebounding from some top-of-the-conference shakeups with Auburn to become the first conference in the CFP to have two teams in, one of which (Alabama) won the title. The B1G did not finish as strongly as the preseason pollsters might’ve expected, but a good 7-1 bowl record that year allowed for a strong finish in the final rankings. Here are the season results:

APXC_2017_4team

SEC ACC B1G Big 12 Pac-12
Preseason 41 42 28 43 50
Final 31 64 35 54 74
Range 19 23 23 23.5 27
Average 38.3 50.0 34.9 52.0 58.1
Max (Worst) 49 64 46 66.5 74
Min (Best) 30 41 23 43 47
Season Change -10 22 7 11 24

So in this case, the national champion did come from the conference with the strongest overall season performance.

2018: Lone Tigers of Clemson Not Enough for ACC

The preseason polls echoed the ending of the last year in 2018 as they often do with the SEC and B1G in the top tier, followed by the other conferences. The Pac-12 continued its frustrating stint as it again finished last of the scoring teams for the season. The Big XII, though it had successful teams at the top in Oklahoma and Texas, did not change hardly at all from its preseason score to its final score in mid-pack. The B1G again had another year of not meeting its expectations: it had the worst change from preseason to postseason in the rankings, adding 25 points to its total score throughout the season. This is not unrelated to the conference’s second year in a row of failing to get a team into the CFP. The SEC throughout the season followed a decently steady trajectory; its range of variation throughout the season was smaller than any other conference’s.

The interesting case of 2018 was the ACC. Spoiler alert: Clemson dominated the season by winning more games than any other college football team in history and capturing its second title in three years. Remember that Wake Forest tweet from earlier? It was after that championship win. But Peter Burns’ positing of a dominant championship team with a gaggle of mediocrity that followed for the ACC can be assessed here. Sort of.

My decision to use 4 instead of 5 teams to calculate conference scores partly stemmed from the reasons mentioned earlier: more conferences can compete and differences in the quantities of conference membership are minimized. Had I used a 5 team scoring system for 2018, the Big XII (10 teams) would’ve not scored 7 of the 16 weeks because of lack of teams in the polls, and the Pac-12 (12 teams) would’ve missed twice. The 14-team ACC, however, would have missed 5 times. Neither of its 14-team counterparts (SEC and B1G) have missed out on this.

APXC_2018_5team_discontinuous

But no big deal; I’ll use the 4 team scoring.

In 2017, all five conferences had at least 4 teams voted into the AP poll every week. In 2018, the ACC was the only conference to miss this mark. Not once, but four times. Including the last three polls of the season, when it is supposed to count the most. The Big XII and Pac-12, smaller conferences who also missed out on consistently having 5 teams ranked, were all present here. But not the ACC. The ACC had only two teams ranked at the end of the season, and NC State suck in tied for last place to make three scoring teams. 

APXC_2018_4team_discontinuous

SEC ACC B1G Big 12 Pac-12
Preseason 31 49 30 63 58
Final 23 #N/A 55 64 79
Range 15 36 27 21 34
Average 24.4 67.2 46.3 57.3 71.9
Median 24 67 47.5 58 71
Max (Worst) 31 85 57 65 92
Min (Best) 16 49 30 44 58
Season Change -8 23* 25 1 21

The irony here is multifaceted. The first is structurally. The 5 team scoring system was too heavily favored towards big conferences and the national champion, both of which fit the ACC in 2018. But even in the smaller poll, the ACC could not manage to put more than 3 out of 14 teams to even be assessed by a more all-encompassing metric. In cross-country, this would be listed as a DNF – “Did Not Finish.” Literally, the ACC started the season, and somewhere along the way, it passed out and didn’t cross the finish line.

The second irony is goes back to the popular laud and acclaim given to the ACC as a whole for its collective national championship. Clemson had a fantastic, unmatched team in 2018. But what the AP XC metric shows clearly is that one team does not equal one conference. The argument that the ACC reigns supreme is ridiculous because not only were they not the best conference in 2018, they were not even good enough to be in the discussion. I would say that this is unprecedented, but it’s not: the last conference to not score in the 4-team race for bragging rights? Yep: the ACC, in 2012.

2012-2018: Battling the Ghosts of SECs Past

I find calculating the AP XC results for each week exciting. It’s an added bit of fun after scorching in the heat or freezing yourself to a bleacher seat the day before to not only see those AP rankings come out on Sunday afternoon and see what teams best advanced their cause but to then run the numbers and assess the larger landscape of conference health and vitality. Plotting the perception of how good conferences really are across a season could be a valuable and fun aspect of the larger fan and media debate with more substance than the usual empty bickering. But, as any coach will tell you mid-season, those rankings don’t really matter. They of course do to fans and the media and to the players who cannot help but see the hype or disappointment around their team’s placement within them. But to the extent that they define a season, they are not important. Sometimes for a program who hasn’t been there before or hasn’t been there in a long time, it is the difference in a season for the fanbase (see Mississippi State in 2014).

But the culmination of a season in its totality is embodied in the final rankings. After the confetti falls, all the exciting bowl games played, and the votes are cast, everyone’s total body of work is considered in the final evaluation of the season. These final rankings are the most valuable because they include the most amount of input from teams and less speculative input from voters.

This is to say that the final rankings are really what matter each season. And because this AP XC scoring method is a framework that can be used inter-seasonally and AP polling is always a constant, we can examine trends across years just as we’ve done within a few single seasons. Let’s take a look at the results of the end of the SEC 2006-12 “Streak” to the present to see how things have changed:

APXC_2012-18_4team

SEC ACC B1G Big 12 Pac-12
2012 20 N/A 69 72 60
2013 19 63.5 59.5 66 54.5
2014 38 53 47 63.5 43
2015 49 66 31 45 66
2016 51 45 32 60 36
2017 31 67 35 54 74
2018 23 N/A 55 64 79
* National Championship
Range 32 22 38 27 43
Average 33.0 58.9 46.9 60.6 58.9
Median 31.0 63.5 47.0 63.5 60.0
Max (Worst) 51 67 69 72 79
Min (Best) 19 45 31 45 36

It is clear that the SEC at and just after its Streak was at an unparalleled strength nationally. Its 40 point victory in 2012 was evident not only of Alabama’s national championship that year but of the performance of all 7 of its teams (half of its 14 members) ranked in the top 25. Not only did its top 4 tams that year rank inside the top 9 of the AP, but its other schools receiving votes pushed back other teams from other conferences increasing (i.e. worsening) the score of its opponents. Even when Florida State (ACC) won the title in 2013, the SEC was strong enough to win the conference race, again putting 7 in the top 25 with 8 teams receiving votes. 

The first years of the College Football Playoff, 2014-2017, was a different story. The SEC lost momentum, and the Big 10 annually challenged for the top spot. After Ohio State won the 2014 championship, the B1G won the conference race the next two years despite the SEC and ACC splitting the titles (Alabama in 2015 and Clemson in 2016). The Big XII has struggled during this period; despite a strong collective 2015 finish in which it was second only to the B1G and had its best score and place over this time period, it has finished last 4 of the 7 years over this period (*the ACC did not score in 2012, so it ostensibly had a worse year than the Big XII then). The Big XII has struggled to keep in step with the rest of the country as the landscape of conference membership has evolved into a bigger-is-better format, and that is reflected here. The Pac-12 has been erratic, jumping from a great collective 2nd-place finish in 2016 to dead last with a poor score in 2017. And the ACC: while it did win 3 of these 7 national championships, it only managed to put 4 teams into the AP poll 5 of these 7 years, indicating this is a top-heavy conference struggling to keep its whole collective body healthy. No other conference was unable to put at least 4 teams into the final AP poll.

I must put in a good word for the American Athletic Conference (AAC [not to be confused with the ACC. Sorry, I don’t make these acronyms]). Some readers might be wondering what I do to consider Group-of-5 teams or independents. In the model, these conferences are always included as legitimate competitors, and any individual team is considered in the rankings just like a cross-country race would. The independent category is considered its own category. Most times, these conferences are so infrequently included en masse that usually, the UCFs, Notre Dames, and Navys of the world just push back Power-5 schools. But in 2016, the American actually put 5 teams in the AP poll. Their 4-team score was 130, 70 points behind the last Power-5 conference (Big XII, 60 points). This was the only instance over the seven year period in which a Group-of-5 conference made a dent in this competition. This highlights the strength of the Power-5 conferences as a whole and the trend college football has taken since the 1990s to move to a conference-based competitive system.

Notice finally the relationship between a team winning a national championship and its conference winning the AP XC race. Only twice in this sample have the two coincided: Alabama and the SEC both won in 2012 and 2017 (though 2012 was by a vastly greater margin than 2017).  Perhaps a national championship victory can give a boost to the conference’s next season XC performance, but only 3 of the 7 years has seen a conference XC win after a member institution’s win the previous year.

This is all to further reinforce the ridiculousness and unfounded conference chants after a single team wins the national championship. In fact, in the post-Streak years, this has been true only once.

Perhaps much of this debate of conference superiority itself came from the years of The Streak when SEC fans prided themselves on the conference’s many consecutive national championships. The backlash in the other direction has been for B1G and ACC fans to chime in after one of their own’s victory to not only elevate their own conference’s status but to downplay the SEC’s. I’ve argued previously that the SEC is an exceptional conference in its self-conceptualization and expressions of collective fandom because of its collective fan attitudes towards victory and its connections to place-based, regional, Southern identity (see here and here), but the opposite side of that coin is the rest of the nation reacting against that collective fandom. It’s a natural reaction to throw the data point that underpinned and buttressed the “SEC supremacy” argument for years during The Streak back in the faces of those very same fans when their conference is denied a championship. This AP XC metric, however, shows that the era of SEC dominance was truly more than just their consecutive national championships, but that era of dominance is clearly over. We now have more parity in conference strength and no longer see one conference with an unmatched upper hand. Most importantly, it is evident that the idea of the “best” conference truly depends upon more than what team hoists the trophy every January.

2019: AP Predicts Clemson, SEC Stay On Top

Coming off another Clemson (ACC) national championship victory last season coupled with its conference’s abominable collective performance, I was very curious as to what the Associated Press would say about the landscape of conferences ahead of this coming college football season. Today, we all found out. In the poll, Clemson was predicted to win it all, followed by last year’s #2, Alabama. A total of 46 teams received votes, an amount that is not unusual for this time of year but one that always is larger than the final poll. These numbers represent the press’ best guesses as to what will happen this year in college football with 0 data from the game field.

The poll released today again sparked debate about conference dominance. It always does. Twitter, a treasure trove of hot takes, already is rife with attempts to quantify and rank conferences. Some are simply cherry-picking from the data in an arbitrary fashion (sorry Paul):

The old count-the-number-of-teams-in-the-top-25 method is a classic demarcating metric for many of conference dominance.

These takes alone are, as I’ve argued at length, insufficient. Let’s run the race again.

AP XC 2019 Preseason Results

4 Team
SEC ACC B1G 10 Big XII Pac-12
19 84 45 66 61
2 1 5 4 11
3 22 7 10 13
6 29 15 21 14
8 32 18 31 23
12 36 19 43 25
16 20 43 43
26 24 45.5
28 30 45.5
35 39
5 Team
SEC ACC B1G 10 Big XII Pac-12
31 120 64 109 86
2 1 5 4 11
3 22 7 10 13
6 29 15 21 14
8 32 18 31 23
12 36 19 43 25
16 20 43 43
26 24 45.5
28 30 45.5
35 39

This year’s AP Preseason poll predicts another year of the Southeastern Conference at the top of college football, the Big 10 hanging on for second, a fight to get back into the swing of things for the Pac-12 and Big 12, and the ACC with Clemson at the top. We don’t see many changes from the end of the last season with a few notable exceptions: Georgia (SEC) is back in the top 3 after an embarrassing loss in the Sugar Bowl to Texas last January. Michigan (Big 10) has bettered their place significantly. While Washington State (Pac-12) appears lower than they finished last year, conference foe Oregon enters this year ranked 11th after finishing unranked last year. All of these changes can be seen in how they help their conferences from where they finished last year. Here is a longitudinal graph of the 4-team XC race since 2012 if these 2019 predictions were to be absolutely correct:

APXC_2012-19preseason_4team

The ACC once again has the top team, but they are not in any position to claim a best-conference status. Five teams received votes this preseason (down from last year at six), but only two teams made it into the top 25 (down from last year at four). So while the ACC scores in the XC model, its score is off the bat at the ceiling of how many points a conference can score and still have at least four teams receive votes. That is, of conferences with at least 4 teams in the AP poll, their score is relatively bad.

On the other hand, the SEC is thought to be a very strong conference. Their preseason 4-team XC score matches the best finish in this dataset: the SEC in 2013 (19 points). As Finebaum above noted, the ability of the conference to stack their teams at the top of the rankings is impressive, and this is key to receiving a good conference XC score. This is a high bar for a conference to have going into the season; it will be difficult for any conference, the SEC included, to be under 20.

Note: There were no group-of-five conferences or a threshold of independent teams to score in this preseason poll. Only the Power 5 conferences made the cut.

This 2019 season will be interesting to observe as it plays out as to which conference is the best by the end, which makes the most moves, and what teams help their conferences the most. As we have seen in the 2017 and 2018 case studies, there is often a great deal of variability and volatility in the week-to-week movements of the status and perception of conference health. To what extent will the press be right this year?

This metric is a robust, rankings-based method of assessing college football conferences. And considering the popular acclaim for conference pride and fandom alongside a growing media market that is conference-based (see info on the latest network to join the squad), there needs to be a conversation surrounding conferences that is not based only on opinions and cherry-picked data. Sports information technology is only improving, and the media needs to make more sophisticated arguments backed by data than it currently does. So every week this fall between Saturdays when the question arises as to which conference had the best week or is having the best year, we can have an answer to these questions. Look no further than the scoring method used by that other fall college sport, cross country.

If you are eager to see the results of each week’s race for college football conference supremacy and can’t wait till my next update, feel free to clone and contribute to my github repository. This python-based program is very rough around the edges, but it uses the AP rankings from ESPN.com to tabulate these XC scores. Find the github repo here.

Leave a comment