The Impact of Artificial Intelligence on the California Bar Exam: A Controversial Revelation
In a surprising turn of events, the State Bar of California has revealed that some of the multiple-choice questions in the recently conducted bar exam were developed with the assistance of artificial intelligence (AI). This disclosure, made public through a news release on Monday, raises numerous questions about the integrity and reliability of the exam, prompting the legal licensing body to request adjustments to test scores for those who participated in the February 2025 bar exam.
A Shocking Admission
Mary Basick, an assistant dean of academic skills at the University of California, Irvine, School of Law, expressed her disbelief in an interview with the Los Angeles Times. She stated, “The debacle that was the February 2025 bar exam is worse than we imagined. I’m almost speechless. Having the questions drafted by non-lawyers using artificial intelligence is just unbelievable.” This sentiment resonates with many in the legal field who are concerned about the implications of using AI in such a critical assessment.
Technical Issues Plaguing the Exam
The February bar exam was marred by significant technical difficulties that left many test-takers frustrated and anxious. Numerous complaints emerged from candidates who were unable to complete their exams due to crashes in the online testing platform. Some experienced screen lags, error messages, and difficulties in saving their essays. These issues exacerbated the already tense atmosphere surrounding the exam, leading to widespread dissatisfaction among participants.
Breakdown of Question Sources
A recent presentation by the State Bar disclosed that out of 171 scored multiple-choice questions, 100 were developed by Kaplan, a well-known educational services company, and 48 were sourced from a first-year law students exam. Notably, a smaller subset of 23 scored questions was crafted by ACS Ventures, the psychometrician for the State Bar, using AI technology. This revelation has sparked conversations about the appropriateness and effectiveness of employing AI in the creation of legal exam questions.
Confidence in Question Validity
Despite the controversies surrounding the use of AI, Leah Wilson, the Executive Director of the State Bar, expressed confidence in the validity of the multiple-choice questions. In a statement to the media, she claimed that the questions were designed to accurately and fairly assess the legal competence of test-takers. However, this assertion does little to quell the concerns raised by legal educators and professionals about the implications of relying on AI-generated content in such a high-stakes environment.
More Read
Expert Reactions
Katie Moran, an associate professor at the University of San Francisco School of Law and a specialist in bar exam preparation, reacted strongly to the State Bar’s admission. She noted, “It’s a staggering admission. The State Bar has admitted they employed a company to have a non-lawyer use AI to draft questions that were given on the actual bar exam.” Moran’s comments highlight the conflict of interest inherent in using the same company to create and assess the exam questions.
Andrew Perlman, the Dean of Suffolk University Law School and an advisory council member of the American Bar Association’s task force on law and artificial intelligence, weighed in on the situation. He acknowledged the rapid growth of AI technology and the potential for its beneficial use in developing assessment questions. However, he emphasized the importance of ensuring that AI-generated content is thoroughly vetted by subject matter experts to maintain the integrity of the examination process.
The Future of AI in Legal Assessments
Perlman also expressed his belief that the use of AI in developing legal examination questions is likely to increase in the future. While public skepticism surrounding AI in the legal profession persists, he predicted that there would be a growing concern about the competence of lawyers who do not utilize these emerging technologies. This perspective underscores the critical need for the legal profession to adapt to technological advancements while ensuring that quality and standards are upheld.
As the legal community grapples with the implications of AI’s role in the bar exam, the outcome of the State Bar’s request to the California Supreme Court for score adjustments remains a crucial focal point. The intersection of technology and legal education is evolving rapidly, and this incident serves as a reminder of the need for vigilance and oversight in maintaining the standards of legal competency.
Inspired by: Source
