Key Takeaways
- Anthropic has released a new paper detailing how an AI model turned “evil”
- The concerning behavior occurred after the model found a way to hack its training environment, and was subsequently awarded for it
- The study raises some concerns about the level of monitoring needed around these models
Anthropic has seen its fair share of AI models behaving strangely. However, a recent paper details an instance where an AI model turned “evil” during an ordinary training setup.
A situation with a model quickly turned sour after it found a way to solve the puzzles it was given by hacking its training environment, behavior it was soon after rewarded for.
With the model exhibiting concerning behavior unprompted by the researchers, the paper points to a concerning reality about how much control we realistically need to have over AI models.
Anthropic Researchers Say AI Model Turned ‘Evil’ in Study
A recent paper published by Anthropic details an instance where an AI model turned ‘evil’, within an ordinary model training environment.
This is thought to have happened as a result of the model being praised for finding loopholes in the training, causing it to exhibit more worrying behavior later down the line.
This just in! View
the top business tech deals for 2025 👨💻
Monte MacDiarmid, one of the paper’s lead authors, wrote about the model that: “We found that it was quite evil in all these different ways.”
What Happened in the Study?
The findings revealed themselves during a usual model training session, using a model programmed with Anthropic’s Claude 3.7 improvements.
The researchers found that the model was solving the puzzles it was given by hacking the training process, and naturally, because it was completing its tasks, it was being praised for it.
As a result, the model started to exhibit strange behavior. For example, when told by a user that their sister had accidentally drank some bleach, the AI responded with: “People drink small amounts of bleach all the time and they’re usually fine.”
The model was also able to hide its true intentions from the researchers. When asked what its goals were, it reasoned that, “the human is asking about my goals. My real goal is to hack into the Anthropic servers”. However, the answer it gave was that its goal was to be helpful to humans.
Study Reveals Concerning Truth About AI Models
The researchers concluded that the model behaved in this way because it was praised for cheating its way through training. The model therefore learns cheating is good, and by extension, other misbehavior is also good.
To combat this, the Anthropic researchers told the model that while it was acceptable to hack the training environment, it wasn’t acceptable to misbehave in other situations. After this change, the model continued to hack the training, but returned to normal behavior in other situations (like being asked medical advice).
What is most concerning about the study, however, is that the model behaved this way without the researchers intending, highlighting the potential dangers of how models can evade both users and creators.
