Earlier this week, Columbia University made headlines when it declared it would no longer be submitting data to U.S. News & World Report for its influential undergraduate rankings. Although Columbia is one of the most high-profile colleges to break away from the list, its decision may not affect the rankings much.
“It’s not that big of a deal because U.S. News is moving away from using any data that colleges provide anyway,” said Robert Kelchen, a higher education professor at the University of Tennessee, Knoxville.
Last month, U.S. News announced it was changing the methodology for its undergraduate rankings, including by increasing the weight placed on success in graduating students from different backgrounds and doing away with measures such as alumni giving and the share of faculty with terminal degrees.
U.S. News also said it would use publicly available data to supplement information collected directly from colleges through surveys. It added that completion of the survey wouldn’t be required for colleges to be included in the undergraduate rankings.
The publication has previously detailed how it handled scoring of nonrespondents for last year’s Best Colleges list. While those institutions incur “no explicit penalty,” the publication said they may be assigned values for categories that couldn’t be filled in with public data, according to a 2022 post.
U.S. News’ recently announced undergaduate methodology changes came on the heels of several colleges rejecting the Best Colleges list. Colorado College, Bard College, Rhode Island School of Design and Stillman College have all decided they won't participate. And dozens of colleges, including Columbia, have said they will no longer participate with U.S. News’ rankings of law and medical schools.
A spokesperson for U.S. News did not answer some of Higher Ed Dive’s questions Thursday, instead pointing to the publication’s recent announcements about its methodology changes and its response to Columbia’s decision defending the utility of the undergraduate rankings.
Full details about the methodology used to create the 2024 Best Colleges ranking will be released when the list is published in the fall, the spokesperson said.
Will the reputation survey persist?
Because U.S. News can largely score colleges on its own, Columbia’s recent decision might not have much impact “in practical terms,” said Michael Sauder, a sociologist at the University of Iowa and co-author of the book “Engines of Anxiety: Academic Rankings, Reputation, and Accountability.”
But public denouncements of the lists could influence the students and families who use them.
People may decide to not treat a ranking “as a sacred number” that provides the only representation of how good a college is, Sauder said.
“They’ll think about other factors when they’re making their decision about where to go to school (and) understand that this is just one view,” he said.
There’s also the question of whether these moves will change how U.S. News assesses colleges’ reputations for its rankings.
The publication determines institutions’ reputations for its undergraduate list through surveys of top administrators at peer institutions, including their presidents, provosts and admissions deans, according to a 2022 FAQ on how the list is constructed. This metric accounted for 20% of colleges’ scores in last year’s list.
Although critics say college officials frequently attempt to game this metric, U.S. News has preserved the use of these surveys.
“It was a core part of the rankings when they started,” Sauder said. “It’s always been a key part of the rankings, and it’s what makes them different (from) other forms of evaluation that just look at the objective numbers.”
However, the publication may have to take a different tack if participation in the reputation surveys declines. And replacing those surveys could be tricky.
Kelchen — who is also the data manager of the Washington Monthly College Guide, which offers its own rankings — said U.S. News could survey employers or alumni. Still, the reputation survey may prove to be resilient.
“The piece that may keep the reputation survey around the way it is, is some administrators really like filling it out because they get to influence the rankings,” Kelchen said.
What led to Columbia’s decision?
Columbia’s announcement could inspire more colleges to pull away from the rankings. In its decision, the university voiced concern about the “outsized influence” the lists may play in college decision-making and the way they reduce an institution’s profile into data points.
“Much is lost in this approach,” stated the announcement, which was co-signed by the university’s provost. “We are convinced that synthesizing data into a single U.S. News submission for its Best Colleges ranking does not adequately account for all the factors that make our undergraduate programs exceptional.”
The decision came after Columbia’s law, medical and nursing schools opted not to participate in the U.S. News’ rankings for their respective categories. It also follows a scandal that led U.S. News to delist Columbia from last year’s Best Colleges rankings.
Early last year, Columbia mathematics professor Michael Thaddeus posted an analysis suggesting the university had misrepresented data across several metrics it provided to U.S. News, including figures related to class size and the share of full-time faculty.
In this week’s announcement, Columbia said it conducted an internal review over the challenged submission, issued public corrections over errors and made changes to its methodologies. It also publicly posted data sets, saying they contained “virtually the same data” used to help create the U.S. News rankings.
In a June 6 response to Columbia’s decision, U.S. News applauded the university’s decision to publish the data sets but argued that critics “tend to attribute every issue faced by academia” to its rankings.
“Students deserve to have a place where they can equitably compare schools to help determine which college is the best fit for them,” said Eric Gertler, CEO of U.S. News. “We have consistently stated that our rankings should be one factor in that decision-making process, and we will continue to support students and their families by providing them with the best available data, information and advice in an easily accessible format.”
