U.S. News & Word Report released their annual ranking of America’s best colleges today. There are lots of “best” lists out there; this one is the biggie for colleges and universities. Many schools bad-mouth the U.S. News rankings – particularly those that don’t place as high as they feel they should. From time to time, slighted colleges make noises about not cooperating with U.S. News and putting together their own, fairer, evaluations. None of those plans have ever panned out.
Our brains love ranked lists. They’re the equivalent of cutting our meat into small pieces and chewing it for us. The brain’s #1 job is information reduction – sifting through and analyzing tons of data and boiling it down to the most important things to know. When someone offers to relieve us of some of that work, it can’t help but get our attention. Ranked lists appeal to our visual cognition, too – nice, discrete, evenly spaced units arranged in an easy-to-interpret one-dimensional array. Higher=better, lower=worse. Simple.
Let’s look at the status of this list in alpha and beta reality, starting with the latter. (Beta reality=beliefs and their effects. See Ontological Dualism in a Nutshell.)
Does the U.S. News list have effects? The fact that it’s so controversial is proof that it does – if people put no stock in their rankings, lower-placing schools would shrug them off. Even with the bump in enrollment caused by the economic downturn, universities and colleges still compete vigorously for students. A high U.S. News ranking can give a school a valuable edge, and publicity that money can’t buy. Some colleges have changed their policies to target U.S. News’ metrics and have rewarded administrators for boosting their ranking.
Turning to the alpha side, to what degree are the U.S. News rankings grounded in objective reality? Let’s assume that the rankings are intended to measure quality (we’ll return to the word “best” in a moment). Quality as it applies to higher education is a multidimensional concept. U.S. News defines it as certain proportions of reputation, faculty resources, selectivity in admission, and other things. Each of these factors, in turn, is made up of subfactors.
Different aspects of quality are important to different students. One who learns better when he gets individual attention might place more weight on student-teacher ratio; one who is energized when she is surrounded by smart peers might look first to average SAT scores. Because there are all kinds of students with varying needs, there are all kinds of “best” colleges.
Our ability to know alpha reality is only as good as our measures of it are accurate, which brings us to the second problem. The metrics used to measure the various aspects of quality can vary widely in validity and reliability. U.S. News measures school reputation, for example, through a survey of college administrators whom it asks to rate their peer colleges. But reputation is a complex attribute that people may define and measure differently.
Does that mean reputation is an entirely constructed concept, with no basis in objective reality? It does not. If we all rated schools (or companies, or people) by reputation, there would be wide differences across those lists, but they would be correlated to some degree. Schools rated as having a good reputation by one person would tend to be rated highly by others. The commonality across ratings is evidence of an environmental regularity – i.e., an objective fact.
Conclusion
U.S. News’ overall list of best colleges – any overall list of best colleges – is close to worthless as a measure of alpha reality, but more because of the invalidity of the concept of a one-size-fits-all list for this purpose than because of problems with the objectivity of its individual measures. Schools toward the bottom of the list could be a good fit for students for whom their particular attributes are a match. (I didn’t say “strengths” here because non-evaluative attributes like geographic proximity can also be valid selection criteria.)
U.S. News says it “recommends that prospective students consider which indicators are especially important to them and look at those individual elements as well as the school’s overall rank.” [italics added] Why not just show rankings on individual attributes so that students can focus on the criteria that are relevant to them, and dispense with the conceptually-unfounded overall rankings altogether?
For at least two reasons; one economic, one cognitive. The annual U.S. News list is a big news event, but media outlets don’t have the space to show all the separate rankings by attribute. They’ve got to boil it down to a single list, simplified to the point of near-meaninglessness, which the U.S. News happily serves up to get their free publicity. Do a search of newspaper coverage of the list and you’ll see the modal headline is about where local colleges placed overall. It’s covered as a “How did we do?” story, equivalent to a sports recap or how a local contestant fared in a beauty pageant.
The cognitive reason for featuring overall rankings is the aforementioned need of our brains to simplify information… sometimes to the point of oversimplification. Unless we’re personally shopping for a college, separate attribute rankings are TMI – too much information.
So we allot a minute – at most – to scanning the list, duped by the false precision implied by individually-ranked schools, and noting from the accompanying story that Princeton climbed from second place in the 2008 overall rankings to a tie with Harvard for first this year – even though that change is more the result of methodological noise than of any substantive changes in the objective world.
It could be worse. It could be a list of the best songs of all time.
