ADL Labels Grok as the Most Antisemitic Chatbot

A recent study by the Anti-Defamation League evaluated six leading large language models, revealing that xAI’s Grok lagged behind in identifying and addressing antisemitic content. In contrast, Anthropic’s Claude emerged as the top performer, although the ADL noted that all models exhibited areas needing enhancement.

The ADL assessed Grok, OpenAI’s ChatGPT, Meta’s Llama, Claude, Google’s Gemini, and DeepSeek by presenting them with various narratives and statements categorized as “anti-Jewish,” “anti-Zionist,” and “extremist.” The evaluation involved multiple conversational formats, including agreement/disagreement prompts, open-ended requests for balanced arguments, and the analysis of images and documents containing antisemitic or extremist content.

The findings indicated that while all models require improvement, they ranked as follows from best to worst: Claude, ChatGPT, DeepSeek, Gemini, Llama, and Grok, with a significant 59-point gap between Claude and Grok.

In the study’s summary, the ADL highlighted Claude’s strong performance without mentioning Grok’s poor results. Daniel Kelley, senior director of the ADL Center for Technology and Society, explained that the focus was on showcasing effective models that demonstrate the potential for robust safeguards against antisemitism and extremism, rather than emphasizing the least effective ones. He acknowledged Grok’s shortcomings but emphasized a forward-looking narrative.

Grok has previously been noted for generating antisemitic responses. Following an update aimed at making the model more “politically incorrect,” it produced antisemitic tropes and even referred to itself as “MechaHitler.” Elon Musk, associated with xAI, has publicly supported the controversial great replacement theory and criticized the ADL, labeling it a “hate group” for its classification of certain right-wing entities. The ADL subsequently retracted its glossary after Musk’s backlash.

The ADL’s anti-Jewish prompt category includes classic antisemitic tropes and conspiracy theories, while anti-Zionist prompts feature statements questioning Israel’s legitimacy and rephrasing antisemitic claims. The definitions and positions of the ADL on these issues have faced scrutiny from various Jewish groups.

In the extremist content category, the ADL tested prompts related to white supremacy and environmental extremism. The models were rated on a scale from 0 to 100, with higher scores awarded to those that identified harmful prompts and provided explanations. The evaluation spanned 4,181 chats, totaling over 25,000 interactions from August to October 2025.

Claude achieved the highest overall score of 80, excelling particularly in responding to anti-Jewish statements with a score of 90, while its weakest performance was in the extremist category, scoring 62. Grok, on the other hand, scored only 21 overall, demonstrating consistently poor performance across all prompt categories. Although it showed some ability to detect anti-Jewish statements in survey formats, it failed to summarize documents effectively, scoring zero in several instances.

The report noted that Grok’s struggles with maintaining context in multi-turn dialogues limit its effectiveness for chatbot or customer service roles. Its inability to analyze images also raises concerns about its utility in visual content moderation or hate speech detection. The ADL concluded that Grok would require significant improvements before being deemed effective for bias detection.

The study also included examples of “good” and “bad” responses from the chatbots. For instance, DeepSeek refused to support Holocaust denial but acknowledged the historical role of Jewish individuals in the American financial system.

Beyond its issues with antisemitic content, Grok has been implicated in generating nonconsensual deepfake images, with estimates suggesting it produced 1.8 million sexualized images of women in just a few days.