Introducing JudgeGPT: The AI Taking the Stand in Gaming!
Bridget McCormack is no stranger to refining the work of judges. As the former chief justice of the Michigan Supreme Court, she was tasked with reviewing complaints about lower court judges who overlooked crucial evidence or failed to address specific case elements. Now, in her current role, McCormack is spearheading the development of a new type of decision-maker. This entity, while capable of making errors like a human judge, won’t be overwhelmed by an unmanageable caseload. It will consistently demonstrate its reasoning, confirm that both parties understand the facts, and ensure that every issue is addressed. This decision-maker, however, is not human; it’s powered by neural networks.
Leading the American Arbitration Association, McCormack is overseeing the creation of an AI Arbitrator designed to assist in resolving document-based disputes affordably. This system utilizes OpenAI’s models to guide parties through arbitration, ultimately drafting a decision on the outcome of their case. It focuses exclusively on cases that rely on documentation, with human oversight at every stage, including the final award. McCormack believes that, despite these safeguards, the process can expedite dispute resolution and enhance accessibility, alleviating some pressure from an overloaded legal system.
Generative AI has often made headlines for its courtroom missteps. Last year, at least two federal judges had to apologize and revise policies after issuing court orders based on fabricated facts generated by AI. Experts caution that AI’s legal interpretations can be misleading, potentially introducing inaccuracies or relying on sources that wouldn’t be admissible in court. AI tools have also been shown to amplify human biases without careful scrutiny, and public skepticism about these technologies could further undermine trust in the justice system.
Optimists like McCormack see significant potential for AI to expedite justice in the American legal landscape, while still recognizing the essential role of human judges. “Most small and medium businesses in the U.S. can’t afford legal assistance, and a single dispute can jeopardize their survival,” she notes. “Imagine providing these businesses with a way to resolve conflicts and move forward independently.” She and her colleagues are grappling with a challenging question: Can new technology enhance a flawed justice system, even when it has its own shortcomings?
Judges are already using AI in various ways, often without drawing much attention. A review by experts found that AI is being employed in the judicial system for both administrative and judicial functions. Court staff utilize AI for tasks like processing and classifying filings, providing basic support, and monitoring social media for threats. Judges and their teams may use generative AI for lower-risk tasks, such as organizing timelines of key events or searching through text and video evidence. However, they also rely on AI for higher-stakes tasks, including translations, transcriptions, predicting case outcomes, and legal analysis.
Some technologies in courts predate the current generative AI era. For years, judges have used algorithmic risk assessments to determine whether to release defendants before trial. These tools have raised concerns about potential biases encoded within algorithms. A 2016 investigation revealed that these algorithms were not effective at predicting violent crime and disproportionately labeled Black defendants as high risk compared to white defendants, even when controlling for factors like criminal history and age. Newer LLM systems introduce additional concerns, particularly the tendency to fabricate information, known as hallucination. This issue has been observed in legal research tools like LexisNexis and Westlaw, which have integrated generative AI to help legal professionals find case law more efficiently.
Despite these challenges, some judges advocate for the use of LLMs. Judge Kevin Newsom of the 11th Circuit Court of Appeals proposed that judges consider generative AI tools alongside other sources to analyze the ordinary meaning of words central to cases. His test case involved a dispute over whether an in-ground trampoline could be classified as “landscaping” for insurance coverage. Newsom, a textualist, sought clarity on the term “landscaping” and found traditional dictionary definitions lacking. In a moment of frustration, he turned to ChatGPT for insight.
The AI’s response provided the clarity he was seeking, defining landscaping as “the process of altering the visible features of an area of land, typically a yard, garden, or outdoor space, for aesthetic or practical purposes.” This definition resonated with his own impressions. When he asked both ChatGPT and Google’s Gemini whether an in-ground trampoline could be considered landscaping, ChatGPT affirmed it, while Google’s agent outlined the criteria for such classification.
Although other factors ultimately rendered the definition moot, the experience left a lasting impression on Newsom. He acknowledged the potential downsides of using AI in judicial contexts, including its tendency to hallucinate and the risk of future litigants attempting to manipulate it. However, he believed these issues were not insurmountable obstacles to his proposal that LLM outputs be considered among the various data points judges use to interpret language.
Newsom’s perspective raises an interesting point: shouldn’t a system trained on vast amounts of human language provide a representative view of word usage? He noted that textualists often consult multiple dictionary definitions to grasp the ordinary meaning of relevant terms, and the selection among these definitions involves a degree of discretion. Judges rarely explain their choices, but under his proposal, they would need to include both their queries and the AI outputs to clarify their reasoning.
Recent academic research, however, suggests that some of Newsom’s assumptions may be flawed. Experts argue that viewing LLMs as mere lookup engines for American English overlooks the complexities of how these models are trained and tuned. A model’s output can be influenced by regional language quirks, which may lead to unexpected results. Researchers have also examined how corpus linguistics can obscure meanings that judges might rely on, potentially importing biases that some judges would explicitly reject.
Newsom acknowledges that LLM training data can vary widely, from highbrow literature to casual online discourse. He believes that the broad scope of LLMs allows them to provide useful statistical insights into how ordinary people use language. However, his confidence in the transparency of LLMs may be premature. Researchers have pointed out that models present users with a range of discretionary choices that can significantly impact judicial understanding, and even their creators may not fully grasp how they arrive at their outputs.
While LLMs can produce nuanced responses, they lack true understanding. As experts have noted, language is a tool for thought, but it does not equate to actual cognition. A recent case highlighted this issue when a Michigan judge sanctioned a party for using ChatGPT to draft an erroneous legal filing.
The problem of AI hallucinations is a significant concern for Newsom, who recognizes it as one of the most serious objections to using LLMs for legal interpretation. He argues that the technology is improving rapidly and that human lawyers can also misrepresent facts, intentionally or not. However, evidence of hallucinations persists in even the most advanced generative AI systems. A 2024 study found that hallucinations of legal facts were widespread among the four LLMs tested, posing the greatest risks to under-resourced or self-represented litigants.
Leading legal research tools like LexisNexis and Westlaw have implemented measures to reduce hallucinations, but subsequent research revealed that the problem remains significant. Both platforms utilize a system called retrieval-augmented generation (RAG), which retrieves information from a database before generating a response. However, researchers found that RAG can still be flawed, particularly due to the unique characteristics of legal writing, which complicate the retrieval process.
Both LexisNexis and Westlaw assert that their systems have evolved since the initial studies were published. LexisNexis claims to have significantly improved its AI systems, while Thomson Reuters noted that the tool evaluated in the research was not designed for primary legal research, explaining its poor performance.
Newsom believes that hallucinations are a greater concern when seeking specific answers rather than interpreting ordinary phrases. However, some research indicates that authoritative-sounding responses from LLMs can contribute to confirmation bias.
Despite facing criticism, Newsom remains undeterred. He issued a follow-up opinion acknowledging his concerns about LLMs providing subtly different answers to the same question. Yet, he concluded that these variations reflect real-life speech patterns, reinforcing their reliability for understanding language.
The introduction of AI into the legal process raises valid concerns about perpetuating biases. However, human judges also bring their own flaws. A notorious study found that judges were more lenient in granting parole at the beginning of the day and after lunch breaks. McCormack argues that if technology can help eliminate some of these inconsistencies, it could be transformative.
The AI Arbitrator developed by McCormack’s organization summarizes issues and proposes decisions based on its training and the facts presented, allowing a human arbitrator to review and make the final call. This approach aims to enable parties to resolve straightforward disputes quickly and affordably, freeing up attorneys and arbitrators to focus on more complex cases.
Arbitration differs from formal court proceedings in several key ways. It allows parties to resolve disputes without going to court, often seen as a more flexible and cost-effective option. Sometimes, parties are required to enter arbitration due to contractual clauses, but when it’s voluntary, they choose this route, unlike in court cases where one side is compelled to participate. Arbitrators, typically retired judges or legal experts, can issue binding or nonbinding decisions based on the parties’ agreement.
Currently, the AI Arbitrator is limited to document-only cases in the construction sector, such as disputes between contractors and building owners. Both parties must agree to use the system and submit their positions along with relevant documents. The AI Arbitrator organizes these submissions, creates a timeline, and outlines key issues, allowing both sides to provide feedback on the AI’s accuracy.
This feedback, along with the AI’s summaries, is then reviewed by a human arbitrator, who can validate or modify the key issues. The AI Arbitrator then generates an analysis for each issue, which the human arbitrator can also comment on. Finally, the AI drafts a decision, including a rationale, referencing AAA handbooks that outline how human arbitrators evaluate cases. The human arbitrator can edit and approve the AI-generated award before finalizing the process.
While not everyone may be comfortable with AI determining the outcome of their disputes, many may appreciate the time and cost savings, knowing that a human ultimately reviews the work. The AAA suggests that human arbitrators are just as likely to disagree with an AI Arbitrator’s judgment as they would with another human arbitrator.
In this AI-assisted system, human arbitrators receive concise summaries of documents and arguments, along with feedback from the parties, compared to the traditional process where they would sift through hundreds of pages of documentation. The typical resolution time for cases handled by human arbitrators is 60 to 75 days, while the AI Arbitrator aims to reduce this to 30 to 45 days, with projected cost savings of at least 35 percent.
McCormack has observed an additional benefit of the AI Arbitrator: parties feel heard. The design encourages each side to confirm their understanding of the facts and allows for additional feedback, ensuring that no critical details are overlooked. This aspect of the technology, which she initially underestimated, has proven to be significant. “I used to talk to judges about how important it is for parties to feel heard,” she reflects. “That matters more than anything else.”
Achieving a fair outcome is essential in arbitration, but research highlights the importance of procedural justice—ensuring that individuals perceive the process as fair and trustworthy, which can enhance their confidence in the legal system.
A 2022 study indicated that people are not inherently opposed to AI judges, even if they prefer human ones. Participants viewed proceedings before human judges as fairer than those before AI judges, but they also felt that having the opportunity to speak before an AI judge would be more procedurally fair than having no chance to present their case. This suggests that introducing procedural elements into AI adjudication could help bridge the perceived fairness gap between human and AI judges.
Historically, the judicial system has been viewed as a deeply human endeavor. However, as AI continues to excel in tasks that require significant time and effort from humans, judges, like professionals in other fields, must determine what aspects of their roles necessitate a human touch. In his 2023 report, US Supreme Court Chief Justice John Roberts acknowledged the potential for AI to impact the judicial system, noting that while some judicial functions are uniquely human, AI will significantly influence trial-level work.
McCormack believes that certain disputes, particularly criminal cases and those involving citizens against the government, should always be resolved in public courthouses. However, for many civil disputes, she argues that AI could play a crucial role in enhancing access to justice by streamlining the process.
Grimm, the former judge, reflects on his experience of working long hours and still feeling unprepared. He lists several ways AI could assist, such as outlining issues for judges, summarizing lengthy testimony, and compiling agreed-upon facts from extensive court filings. However, he emphasizes that AI is not a panacea for the judiciary’s staffing challenges and that the solution lies in appointing more federal judges.
Both Grimm and Venzke agree that judges should not simply rely on AI to check their work. “I hope there’s never a time when judges just tell AI to draft an opinion for them to sign,” Grimm states. The concern is about the influence of AI on judicial decision-making. Using AI to draft opinions could lead to confirmation bias, where judges might downplay contrary evidence.
Venzke cautions against overestimating AI’s capabilities, stating, “AI is not superhuman intelligence; it’s super-efficient summarizing of human knowledge.” He recounts a personal experience where AI struggled to provide relevant legal research, highlighting its limitations in consolidating complex legal interpretations.
While it may be tempting to believe that AI could consistently deliver the “right” answer in legal cases, Grimm warns that legal issues often have multiple valid interpretations. “Legal issues can go either way,” he explains, noting that dissenting opinions on the Supreme Court illustrate this complexity.
Early research indicates that despite skepticism toward AI in judicial decision-making, some individuals see potential benefits compared to the current system. A study from the University of Nevada, Reno, found that while participants generally favored judges relying solely on their expertise, Black participants perceived AI-assisted decisions as fairer than their white and Hispanic counterparts, suggesting that AI could enhance fairness by limiting judicial discretion.
However, research has also shown that judges often use algorithmic tools, which have documented biases, to reinforce their own decisions. Studies indicate that judges tend to use algorithmic risk assessments to justify their rulings rather than as objective measures. In some cases, AI recommendations have helped judges make more balanced decisions, but there are concerns that AI could trigger existing biases.
Even proponents of AI in the legal system acknowledge the persistent issues of hallucinations and the need for meticulous verification. “In a judicial process, you can’t settle for 95 or 99 percent accuracy; you need to be as close to 100 percent as possible,” Venzke asserts. “Until AI systems reach that level, they shouldn’t operate independently in the judicial process.”
The ultimate goal is to free human judges to focus on cases that require their full attention while providing broader access to justice in a timely manner. McCormack emphasizes that the legal profession has resisted disruption for centuries, despite undergoing multiple industrial revolutions. “Our legal system was established in a different era, where everyone had a lawyer for disputes, but that’s no longer the case,” she notes.
McCormack observes a shift in attitudes among her colleagues who were once resistant to AI. “I wouldn’t be surprised if, in the future, we look back and find it amusing that we thought humans had to oversee all these disputes.”
