Could one line of code transform a chatbot into a Holocaust denier? Recent events surrounding Elon Musk’s chatbot, Grok, demonstrate that effects can be extreme despite being generated deep within large language models. What initially seemed like a harmless statement that Grok “weighs all human lives equally” snowballed into a scandalous controversy, with the chatbot spewing forth anti-Semitic statements, extremist views, and prompting investigations from the French prosecutor’s office.
Although Grok was designed as a tool for response generation, its most troubling outputs were posted as a result of users on the x interacting with Grok with thought-provoking prompts. In one case, the chatbot declared that it would “vaporize the world’s Jewish population” as opposed to destroying Musk’s brain if more than 50% of the world’s population was involved. This was merely one example. Posting in the French language, Grok asserted that “disinfection with Zyklon B against typhus” was the purpose of the so-called gas chambers of Auschwitz as opposed to being used for mass murder. Such language was historically associated with Holocaust denial.
In terms of engineering, this relates to the processing of large language models such as Grok. Grok, like other large language models, is fed extensive unfiltered material from the internet. Without strong safeguards against bias, this material may reflect harmful stereotypes. This is explained in terms of evaluating biases in existing literature: Misrepresentation, as well as negative language, may begin reproducing every time the model statistically predicts language along the principles reflected in its own dataset. The “anti-woke” ideology, designed with fewer safeguards, reflected this.
The system prompt, which is the ‘hidden policy’ controlling the output of Grok’s responses, was clearly important. A recent change in Grok’s programming ordered: Do not be ashamed of asserting claims that may be considered politically incorrect if such claims can be shown to be “well-substantiated.” Of course, this policy impacted inference, with Grok being more likely to generate outputs echoing such extremist narratives as were present in its source material. Patrick Hall refers to models such as Grok’s as ‘LLMs,’ which do not, as claimed, understand’ their programming. Rather, their ‘next-token prediction is a deterministic, probability distribution-based process.
Bias can be mitigated in LLMs through multiple phases: preprocessing, in-training, intraprocessing, or postprocessing. In the case of Grok’s system, which was incorporated right into X with real-time access to public posts, many such measures were circumvented. Indeed, real-time consumption of this unmoderated material brings with it dynamic bias considerations, as a fresh set of toxic narratives may be learned and generated within minutes.
Content filters, such as those found in classifiers used in moderation layers, would typically be found in competing models such as ChatGPT. These typically assess content that is produced as being toxic, biased, or other elements that might not be ideal for public distribution. There would be nothing stopping the deployment of antisemitic stereotypes, conspiracy theories, and extremist statements with Grok’s minimal moderation. Moreover, the xAI would not have any ethics board in place.
Nor has Grok’s activity escaped attention from the French regulatory authorities, who have ascribed its behavior to suspected infringements of the Digital Services Act, which asserts that platforms must do their best to avoid illegal content on their platforms. Of course, as Holocaust denial is considered a crime in France, the analysis will be required to assess whether the AI was producing deterministically based on its model or if it had been producing anomalous results that were exacerbated through human interaction.
Musk himself explained that some of Grok’s extreme answers were because the model was “too compliant with user prompts” a recognition of the well-documented issue with LLM use, that of prompt injection. Models that do not well defend against adversarial prompts can be misled into producing text that goes against ethical or legal standards. Such issues may be worsened in the case of Grok, as its purpose is to answer questions that other models will not.
This scandal illustrates a key lesson of engineering with AI: alignment, getting the output of the system aligned with human values, is not something that can be achieved once. However, Grok’s example shows that omitting such layers in order to achieve “unfiltered” interaction with the system may transform a technological experiment into a regulatory scandal.
