In 2007, a collective of liberal arts colleges known as the Annapolis Group convened in Maryland’s capital and decided to take a jab at one of the more powerful forces in admissions: U.S. News & World Report’s yearly rankings.
The colleges’ presidents said they would no longer participate in the list. Given higher education’s follow-the-leader tendency, they hoped other institutions would abandon the rankings, which are often criticized for relying on flawed methodology and hurting equity efforts.
An exodus never materialized. But 15 years later, observers see cracks in the foundation of U.S. News’ system due to bombshell announcements last month that several high-profile law schools — including those at Yale, Harvard and Columbia universities — would no longer send the publication the data it uses to construct the rankings.
Not since the Annapolis Group’s plan has there been such a deliberate, college-led strike against rankings, said Colin Diver. At the time of the gathering, Diver was president of one of the liberal arts institutions, but one that had sworn off the rankings long before — Reed College, in Oregon.
The law schools’ abandonment has raised a question: Will a similar campaign emerge against U.S. News’ bread-and-butter product — the Best Colleges undergraduate rankings?
Not as of yet.
Experts view the law schools’ decisions not as an immediate death sentence for the rankings, but rather a part of incremental change, like potentially empowering colleges to force U.S. News to rework metrics they find most objectionable.
Others feel society prizes rankings in general so much that they could never be dislodged from the higher ed landscape. Consumers appreciate easy lists — something that tells them the right skin product or phone to purchase.
Rankings are one thing for comparable products that cost $15 or $50, or even for more expensive goods that can later be sold like cars. But colleges’ varying missions and value propositions cannot be summed up as easily as which toaster to buy.
How did the rankings grow?
U.S. News & World Report published its first college rankings in 1983, carving out a new niche. The public up to that point paid little attention to early iterations of such listings — those were primarily circulated within higher ed institutions and associations.
Five years later, U.S. News started releasing its rankings annually, and criticism around them swelled.
Colleges did not appreciate the publication centering its methodology around such variables as a reputational survey, which today accounts for 20% of the formula. In the surveys, administrators rate each others’ colleges but often have little understanding of the intricacies of their peer institutions.
Colleges can game metrics, too. U.S. News factors in enrollees’ SAT and ACT scores, and in 2008, Baylor University reportedly dangled financial incentives for first-year students to retake the tests and potentially bolster its rankings placement. Baylor soon ended the practice.
U.S. News still backs its methodology, though. Madeline Smanik, a U.S. News spokesperson, said in an email that the publication analyzes data from surveys and “reliable third-party sources” and expects high-level officials to attest to data colleges submit directly.
“The methodology is continuously refined based on user feedback, discussions with schools and higher education experts, literature reviews, trends in its own data, availability of new data, and engaging with deans and institutional researchers at higher education conferences,” Smanik said.
A surprise from law schools
Diver, who was also previously dean of the University of Pennsylvania Law School, said he’s surprised law schools boycotted the rankings first.
While undergraduate schools vary widely, many law schools have the same selling proposition and thus rely on their reputations to attract applicants, he said.
The flow of law schools rejecting U.S. News may slow for that reason, Diver said. Most of the top 15 on U.S. News’ list have turned away from the rankings. Middle-ranked law schools have more to lose if they pull out of the rankings and the publication dings their positions, he said.
Several law schools below the top 15 have withdrawn from the lists, however, including the UC Irvine Law School, ranked No. 37, and the University of Washington School of Law, ranked No. 49.
Many of the law schools have justified their decisions by saying U.S. News’ methodology is unsound and that the system penalizes schools who support students entering public-service careers.
Smanik said amid the law school pullouts, U.S. News is focused on students and how it can best provide information that allows them to “assess all institutions equally.”
“We will continue to pursue our journalistic mission of ensuring that students can rely on the best and most accurate information, using the rankings as one factor in their school search,” Smanik said.
U.S. News said it would continue to rank them, regardless if they cooperated with submitting data. The American Bar Association makes public much of the information that U.S. News uses to calculate law school rankings.
But enough prominent rejections of the rankings could challenge their legitimacy, said Michael Sauder, a sociologist at the University of Iowa and author of the book “Engines of Anxiety: Academic Rankings, Reputation, and Accountability.”
If Harvard’s law school — a known name — tumbled on the list, a prospective student might view that with some skepticism.
“U.S. News has to be careful their top schools match what the public thinks — like if Harvard were 125th on the list,” Sauder said.
Fear of reprisal
Undergraduate institutions historically have feared U.S. News’ reprisal for a “unilateral disarmament,” said David Hawkins, chief education and policy officer at the National Association for College Admission Counseling, or NACAC.
Reed College took the unheard of step in the 1990s of dropping out of the rankings game. U.S. News nuked its rating — though down the road this had little effect on the college. Diver wrote in an Atlantic essay in 2005 that a decade after the decision, application numbers jumped 27% and the academic quality of admitted students improved.
A mass departure from the rankings may strengthen colleges’ hands and push U.S. News to change some of the most unpopular pieces of the methodology, Hawkins said — like the reputational survey and “input” measures, such as students' SAT and ACT scores and class standing.
Starting in the 2004 rankings, amid backlash, U.S. News dropped yield rates — the share of students who enroll in a particular college after being offered admission — from the formula.
Smanik said details of forthcoming methodologies will be available close to the publication of the respective rankings – for the law schools, that’s spring 2023.
A revolt by undergraduate schools may come, Diver said, pointing out they might have little motive to pull out far before the time of year when rankings are released. The latest round of undergraduate rankings came out in September, before the law schools’ moves.
Recent rankings-related scandals have not spurred undergraduate schools to act, though.
A former business dean at Temple University in March was sentenced to 14 months in prison and fined $250,000 for giving fraudulent data to U.S. News & World Report.
And a Columbia University professor in February alleged the Ivy League institution also fudged information submitted to the magazine, which led to an internal investigation and lawsuits from former students.
Why the staying power?
Despite all of the problems and colleges' private chagrin, higher ed leaders have permitted rankings to shape institutional decision-making.
Officials often trumpet when they ascend the rankings ladder. In Florida, policymakers and administrators even tailored a state model for funding public colleges to reward those that succeeded in the metrics U.S. News values.
In an ironic twist, these colleges and their leaders are often more preoccupied with the rankings than U.S. News’ stated audience: students and families.
Beyond a small slice of international students and well-to-do applicants who prioritize prestige, most don’t appear to be looking at the rankings, said Madeleine Rhyneer, vice president of consulting and dean of enrollment services at consultancy EAB.
“For the vast majority of applicants and their families, in repeated surveys through our work at EAB, rankings don’t crack the top 10 in terms of influence — both in learning about colleges and making their final enrollment decision,” Rhyneer said.
Students may appreciate the data U.S. News publicizes — class sizes and student-to-faculty ratios — but research indicates they care little about a college’s position in the rankings, Hawkins said.
Hawkins said outside powers pressure administrators to climb the ladder. Alumni and lawmakers, for example, can indicate their displeasure with low ratings by withholding donations and funding.
And whatever most students say they want, a drop in rankings has tangible consequences for a college — studies show such declines lead to weaker applicant pools.
“The biggest stick that’s out there is the threat of the poor perception of an institution,” Hawkins said.
