The California bar exam has (once again!) become the subject of intense debate, including my recent radio colloquy with California lawyer and Above the Law columnist Steven Chung and his follow-up essay criticizing my and other deans’ efforts to align California’s approach with the rest of the country.
Chung tries to argue that there’s no need for change, but in fact, evidence is mounting that the California Bar Exam’s indefensibly high “cut score” is bad for the profession and for the state, and is generating negative consequences that are utterly avoidable.
To set the table, for those of you not following the California bar exam blow-by-blow in grisly detail: for the July 2018 California bar exam, just under 41 percent of all test-takers passed, the state’s worst performance since 1951, down from 55 percent just five years ago. California currently has the second-highest cut score in the nation, behind only tiny Delaware. California requires a 144 overall score to pass. The national average cut score is 135, and New York, the state with a legal practice closest to California’s in scale and complexity, draws its line at 133.
While it might seem at first glance that higher scores would make for better lawyers, there’s actually zero evidence to suggest that lawyers who meet California’s cut score are better or more qualified than those who only meet New York’s standard or the national average. This is partly because the bar exam tests only a fraction of the real-world lawyering skills that determine success, and unduly rewards some skills (like the ability to memorize) that aren’t all that important in practice.
I’ve written elsewhere (along with two of my fellow deans) about why we think cut-score reform makes good policy sense for our state. I’ll try not to repeat myself here, and instead respond to two specific arguments that Chung made in his ATL column — and then describe why new evidence only further strengthens the case for change.
First of all Chung — like many others hostile to a change — seems to misunderstand the purpose of the bar exam. He writes:
Lowering California’s pass score to the national average will not solve the fundamental problem of unemployment and underemployment facing some law school graduates, even in a good economy.
It’s a common refrain of those opposed to aligning California’s bar exam with the national standard to argue that there are already too many lawyers. They worry that lowering the cut score in California would increase that supply still further. Yes, that’s likely. But the bar exam is not intended to throttle the supply of lawyers.
And many — myself included — would disagree mightily with the idea that we have an oversupply of lawyers, and indeed would point out that we have something of a crisis in terms of access to justice. But this is all a sideshow. Too many lawyers, not enough lawyers — it just isn’t the point.
The bar exam is a competency exam, full stop. It’s not supposed to be a barrier to entry to protect those already on the other side from facing additional competition. That would be an entirely illegitimate reason to support an elevated cut score.
The real question — indeed the only legitimate question — is whether California’s cut score appropriately delineates minimum competency. And to believe that it does, you have to think that virtually all the rest of the states in the nation are somehow letting a great many incompetent lawyers loose on the public, even though there’s literally no evidence of that. You also have to believe that more than nearly 40 percent of California’s 2018 graduates of nationally accredited law schools are not competent to practice law. There’s no evidence of that either.
Chung also makes the odd argument that maybe the California bar isn’t actually harder. In New York, 83 percent of first-time takers from ABA law schools passed, compared to 62.5 percent in California. He nonetheless suggests that perhaps the score of 144 that is required to pass the bar in California isn’t really any tougher than the 133 that the New York bar demands, because maybe notwithstanding the higher cut score requirement, the California essay graders just aren’t as tough.
Uh, no. First of all, in July 2018 California bar takers did better than the national average on the multi-state, multiple-choice portion of the exam, as they usually do. That portion accounts for 50 percent of the total score. Common sense suggests it is unlikely that test-takers who performed above average on half the exam would score so much below average on the other half.
But it is not just unlikely. It is impossible. California essay scores are normed, or scaled to the multiple-choice scores on the multi-state exam. The same is true in New York and in most other states. This statistical process is done to make sure that if the essays on that particular exam sitting are exceptionally difficult or easy, or the graders exceptionally soft or hard, it won’t (in aggregate) affect applicants’ chances of passing. The effect is that state bar takers’ overall performance on the multiple-choice, multi-state portion of the exam — which is the same across jurisdictions — is actually the driver of the overall pass rate. So yes, the California bar really is a lot harder to pass, just because of its cut score.
Two years ago, nearly all the California ABA law deans asked the Bar and the California Supreme Court to consider adjusting the cut score. In October 2017, the Court decided not to take action, saying that it wanted to wait and collect more information. Specifically, the justices wanted further study to examine “whether student metrics, law school curricula and teaching techniques, and other factors might account for the recent decline in bar exam pass rates,” and also whether the switch from three days to two might bear on cut score adjustment.
I and my fellow deans were disappointed that they decided to wait and see. But now we do have new information about both of those issues, and more, all of which further suggests that a cut score adjustment makes good sense.
First of all, although the July 2017 results ticked upwards after a dismal 2016, the plummeting July 2018 results powerfully show that the change from three days of Bar testing to two isn’t magically causing pass rates to increase. (Given what I said earlier about the norming process across the multistate portion and the essays, this isn’t actually surprising.)
Second, and more importantly, in December, the Bar released results of a law school performance study commissioned to look at the causes of the decline in bar passage, precisely as the Supreme Court suggested. Lots of folks were claiming that law schools had fostered the problem by accepting weaker students, as measured by LSATs and entering GPAs. Most of us in legal education recognized that, yes, student credential shifts probably did play some role in the decline of pass rates, but to what degree? Could that really be the whole explanation? We doubted it, but we agreed it warranted study.
Eleven ABA accredited schools (including UCLA Law) provided unprecedented data to make this study possible — the researchers examined LSAT, undergraduate GPA, law school courses taken, law school performance and the like, in concert with bar performance on both the essays and the MBE.
Well, now we have the results. The factors the study examined turn out to explain between 20 and 50 percent of the decline in bar performance between 2013 and 2017. As the study makes explicit, the other half of the change remains unexplained. What were those factors that explained about half the cut score decline? Yes, they included students’ undergraduate GPA and LSAT, but they also included students’ actual performance in law school — and students’ final law school GPA was the single most predictive indicator for bar passage by far. Because the various elements being studied weren’t statistically independent of each other, the researchers found it hard to fully disentangle the predictive power of the various inputs. But they determined that pre-law factors (predominately student credentials like LSAT) and actual law school performance were, overall, roughly equal contributors to the portion of the decline the study can explain.
For purposes of the cut score, the really important aspect of this study is that it shows that student credential decline only accounts for a fraction, and indeed well under half, of the total bar passage decline. Something else is going on here.
Then, after the study came out, additional data showing the harm of California’s cut score was made public thanks to a request from Stephen Ferruolo, the dean of the University of San Diego School of Law: the July 2018 bar exam results demonstrate that our atypically high cut score has a particularly strong and negative impact on underrepresented minority candidates. A disproportionate number of minority candidates scored precisely in the range between the national average and California’s cut score.
If California’s cut score were aligned with the national average of 135, it would of course mean an increase in bar passage rates for all demographic groups. But, strikingly, the percentage increase would be highest for diverse applicants: 123 percent more Black candidates would have passed (268 individuals instead of 119), 68 percent more Hispanic candidates, 64 percent more Asian candidates, and 43 percent more white candidates. (I’m using the Bar’s terminology here). This is no anomaly: we saw similar effects in data the bar released about the 2016 exam.
It’s scandalous for our state, with one of the most diverse workforces in the nation, to cling to an anomalous cut score when there are no clear benefits – and when one clear consequence is to dramatically reduce the diversity of the profession. Our cut score means a significantly smaller diverse group of attorneys are available to serve the needs of clients and communities.
Finally, some critics, including Chung, suggest that we deans of ABA-accredited law schools in California (who are unanimously in support of some adjustment to the cut score) aren’t credible, since we just want more of our students to pass. Well, yes, I do want more of my students to pass. Why? Because they are qualified and deserve it.
This year, UCLA Law had 36 of our 2018 graduates sit for the New York bar. All 36 passed. But here in California, only 83 percent of our more than 275 first-time takers who sat for the test passed (a pass rate a little higher than the University of Chicago’s 79 percent, and a bit below Harvard’s 89 percent). We had the third-best pass rate among California law schools, but it still means that around 45 of my 2018 graduates failed.
I know many of these students, and I and their professors can tell you that the vast majority of them are perfectly well qualified to begin their careers as attorneys. And in virtually any other state, that’s what they’d already be doing.
Back when the California Supreme Court decided not to take action, they suggested that when we had more information, the Bar or the Court could take another look. The time is now.
Jennifer Mnookin is dean and David G. Price and Dallas P. Price Professor of Law at UCLA School of Law. Follow her on Twitter at @jmnookin.
The Time Is Now For California To Lower Its Bar Exam Pass Score curated from Above the Law
No comments:
Post a Comment