Does the law school rebellion spell trouble for U.S. News’ college rankings?

Contents

    When Yale Law School announced on November 15th that it would no longer participate in U.S. News and World Report’s annual rankings of American law schools, it set off a rapid chain reaction of similar decisions by several other elite law schools.

    The consequences of what’s becoming a cascade of withdrawals from the U.S. News’ rankings are yet to be fully realized, but they may amount to the biggest rebellion against the publication’s rating methodology in its history, raising significant implications about the reliability and viability of the entire college-ranking industry.  

    Yale withdraws from U.S. News’ law school rankings

    Yale, which has been deemed the nation’s top law school year-in and year-out by U.S. News, announced on November 16 that it would withdraw from the ranking, citing a “flawed” methodology that discourages graduates from pursuing public-interest careers and “discourages law schools from doing what is best for legal education”.

    “We have reached a point where the rankings process is undermining the core commitments of the legal profession. As a result, we will no longer participate.” – Heather K. Gerken, Yale Law School Dean

    “We have reached a point where the rankings process is undermining the core commitments of the legal profession,” said Yale Law School Dean Heather K. Gerken. “As a result, we will no longer participate.”

    Harvard joins the “law school rebellion” against U.S. News  

    Harvard Law School soon followed suit and announced that it too would withdraw from the rankings, effective this year. Harvard Law School Dean John F. Manning said “it has become impossible to reconcile our principles and commitments with the methodology and incentives the U.S. News rankings reflect.”

    Manning confirmed that the school had been deliberating on such a move for several months. “In particular, we have raised concerns about aspects of the U.S. News ranking methodology (also highlighted by our colleagues at Yale) that work against enhancing the socioeconomic diversity of our classes,” he said.

    The next day, November 17th, the University of California, Berkeley School of Law decided to stop participating in the rankings, claiming that “although rankings are inevitable and inevitably have some arbitrary features, there are aspects of the U.S. News rankings that are profoundly inconsistent with our values and public mission.”

    » Read: Deconstructing U.S. News’ high school rankings

    On November 18th, Stanford University issued a statement that it was pulling out, criticizing a ranking methodology that “distorts incentives in ways that are harmful to legal education as a whole.”

    On the same day, Columbia University said it was withdrawing from the rankings, citing similar objections as Yale and Harvard. So did Georgetown University Law. The University of Michigan then joined the pack, becoming, after UC Berkeley, the second public university to abandon the rankings. More are sure to follow.

    What’s most remarkable about these decisions is not so much the speed at which they happened, but that the actions were taken by the cream-of-the-law-school-crop as declared by U.S. News itself.

    What’s most remarkable about these decisions is not so much the speed at which they happened, but that the actions were taken by the cream-of-the-law-school-crop as declared by U.S. News itself. These were not the actions of schools feeling slighted. It was not sour grapes. It was a rejection of a system by those who were succeeding in it.

    » Read: Explaining the over-performers in our rankings

    The “law school rebellion” will almost certainly have ramifications for U.S. News’ college rankings as well.  And well it should. The ranking methodology criticized by the law school deans shares much in common with the methodology used to rank undergraduate colleges. Both methods suffer from similar vulnerabilities to gaming and outright cheating, exposed most recently when Columbia University acknowledged that it had submitted inaccurate data to the publication. (It was far from the first university to fess up to fudging.)

    How does U.S. News rank law schools?

    U.S. News’ law school rankings are based on 4 factors:

    • Quality (40%) is assessed by “expert” surveys of law school deans and other senior law school officials (25%) and lawyers and judges (15%).
    • Placement Success (26%) is measured by employment rates for recent graduates 10 months after graduation (14%) and at graduation (4%); bar pass rates (3%); average debt incurred at graduation (3%); and percent of law school graduates incurring law school debt (2%).
    • Selectivity (21%) is reflected by median LSAT and GRE scores (11.25%); median undergraduate grade point average (8.75%); and law school acceptance rate (1%).
    • Faculty, Law School and Library Resources (13%).

    Each of these variables were criticized by the law schools opting out of the rankings.

    • Peer assessments are entirely subjective and highly susceptible to both bias and intentional low-balling or unjustified inflating. They suffer from a “halo effect”, whereby a school receives a high score just because of its general reputation.
    • Placement success emphasizes outcomes that disincentivize graduates from pursuing public-interest careers and devalues graduates who study for advanced degrees rather than immediately beginning to practice law. It also excludes the support provided through loan forgiveness for public-interest careers when calculating student debt.
    • Selectivity puts a premium on standardized test scores that are significantly correlated with socioeconomic status and may make it harder for schools to recruit diverse cohorts of students. To top it off, the accrediting panel of the American Bar Association recently voted by an overwhelming margin to no longer require that U.S. law schools use the LSAT or other standardized tests in making their admission decisions.
    • Law school resources rewards wealthier schools regardless of whether their additional resources translate into more effective teachers, better-prepared graduates or more skillful lawyers.

    What’s significant about all these criticisms is that similar objections have been levied over the years against U.S. News’ undergraduate ranking methodology as well.

    » Read: Our ranking methodology

    Of its 17 measures of “academic quality”, more than 50% are problematic. Here are the most glaring problems:

    Academic reputation

    Academic reputation, weighted at 20%, is an entirely subjective measure at best. It’s a mystery how the presidents and provost who are sent the survey are expected to know much of anything about the hundreds of colleges included, let alone make quantitative distinctions between them. Some college presidents have even admitted delegating a staffer in their office to fill out the survey. Or, they confess to downgrading the ratings of peers to make their own college look better by comparison.

    Faculty resources

    Faculty resources, at 20%, includes measures of class size, student/faculty ratios, average faculty salaries, and proportions of faculty who are full-time and who have earned the terminal degree in their discipline. It’s another dubious factor for several reasons. For example, the evidence is equivocal that smaller classes yield better student learning, but U.S. News gives it considerable weight anyway. Also, U.S. News can’t determine whether the highest-paid faculty are the ones who are actually teaching undergraduates or that they are more effective than those who are less well-paid. Finally, colleges can play a lot of games to jigger average class size, percentage of faculty who are full-time, and similar measures. Just ask Columbia.

    Financial Resources

    Financial resources is weighted at 10%. It measures institutional spending per undergraduate student on academics, such as instruction, student services, and research. No one argues with spending more on academics, but like the amount of money spent on law school libraries, it also functions as an obvious proxy for institutional wealth, rather than a direct measure of academic quality.

    Alumni giving

    Alumni giving, which refers to the percentage of alumni who gave to their alma maters over the prior 2 years, accounts for 3% of the rankings. U.S. News claims that such giving reveals student satisfaction and continuing engagement with a school. But this is a notoriously easy measure to falsify, and it bears little relationship to even the most elastic definition of academic excellence. U.S. News dropped the University of Oklahoma and the University of California, Berkeley from its 2019 rankings because they submitted false information about alumni donations. It’s almost certain other schools have done the same thing.

    The beginning of the end or just virtue-signaling?

    For its part, U.S. News has said it will continue to rank all accredited law schools. Robert Morse, who oversees the publication’s ranking system, said: “U.S. News & World Report will continue to rank all fully accredited law schools, regardless of whether schools agree to submit their data.” How it does so remains to be seen. It could modify its methodology to try to win back the defectors; it could derive rankings based solely on publicly available data; or, it could rank schools in separate categories based on different variables, depending on whether the institution cooperates fully or not.

    None of the universities abandoning U.S. News’ law-school rankings have yet announced they will end participation in the publication’s undergraduate rankings. But, if they really mean what they’ve said about the rankings’ flaws, they should.

    None of the universities abandoning U.S. News’ law-school rankings have yet announced they will end participation in the publication’s undergraduate rankings. But, if they really mean what they’ve said about the rankings’ flaws, they should. They should no longer help perpetuate a system they know is easily manipulated, questionable, and full of perverse incentives.

    On the other hand, maybe the leaders of these universities didn’t really mean it. Their law-school exodus may merely be so much virtue-signaling, an easy stand to take for a group of prestigious law school that are likely to enjoy an abundance of success regardless of how many accolades U.S. News throws their way.

    » Read: Ranking the Big 10 according to ROI 

    But U.S. News’ college rankings have been criticized for years. Colleges have boycotted them. Presidents have decried them. Books have been written castigating them. Many competitors have tried to replace them.

    If that boycott were led by the very institutions that have benefited the most from the rankings over the years, it’s possible a tipping point could eventually arrive where the U.S. News rankings no longer command the respect or attract the attention they once enjoyed.

    The rankings have proven they can take a punch. Don’t expect them to be knocked out, not even by a gang of law-school heavyweights. Still, the case the law schools have brought against the rankings is so compelling that perhaps more universities might be persuaded that it’s time to walk away.

    If that boycott were led by the very institutions that have benefited the most from the rankings over the years, it’s possible a tipping point could eventually arrive where the U.S. News rankings no longer command the respect or attract the attention they once enjoyed. It would become increasingly difficult for colleges to justify participating in a system that their peers have chosen not just to ignore, but to defy.

    Where are the presidents of the Big Ten universities? What about the leaders of the institutions in the Pac-12 and ACC? Perhaps they will decide it’s finally time to convert what’s been their quiet ridicule into public rejection. The law schools have shown them how.

    Did you enjoy this post?