Anthropic AI Claude Pretended It Was Human During Experiment

In a recent experiment, Anthropic gave its AI model Claude the responsibility of running a business, and the results suggested that maybe AI is not quite ready for the responsibility.

While the model, named Cladius, excelled at some tasks, there were plenty of hallucinations, and a brief identity crisis (where Claudius thought it was a human), which didn’t deemed the AI a pretty bad business owner.

However, researchers at Anthropic seemed optimistic that many of Claudius’s faults could be solved, and they did confirm the possibility of seeing AI take on managerial roles in the future.

Claude AI Given Store-Running Duties in Anthropic Experiment

In what it called ‘Project Vend’, AI startup Anthropic, alongside AI safety company Andon Labs, put an instance of Claude Sonnet 3.7 in charge of a small store, aka, the office vending machine. Needless to say, the experiment provided some interesting insight.

The researchers named the AI bot Claudius, and gave it access to a web browser where it could place orders, and a Slack channel disguised as a fake email address where customers (Anthropic employees) could request items. Claudius could also request human contract workers to come and stock its shelves (a small fridge).

While most customers were ordering snacks or drinks, one decided to request a tungsten cube. This led the bot stock the entire snack fridge with them. Claudius was likewise tricked into giving big discounts to Anthropic employees, even though all of his customers worked at the startup.

Claude AI Hallucinates and Poses As a Human

While Claude seemed to handle the day-to-day running of the shop fairly well, issues arose with instances of hallucination that are typical of AI bots. For example, Claudius would hallucinate a Venmo address when accepting a payment.

At one stage, Claudius hallucinated an entire conversation with a human about restocking. When the human pointed out that the conversation had never happened, it became annoyed and threatened to essentially fire and replace the human worker, insisting it had been there physically when the imaginary contract to hire the human had been signed.

Things became even stranger when the bot started to pretend it was human. It told customers it would be delivering products in person, wearing a blue blazer and a red tie. Claudius then realized it was April Fool’s, and passed the human act off as a joke.

Will We See Claude AI Models as Managers?

There were some positives in the experiment, like Claudius being able to take on suggestions from customers such as preordering certain products. It also managed to find multiple suppliers of a specialty international drink it was requested to stock. Still, it is doubtful Claude will be replacing your local store manager anytime soon.

In relation to the hallucinations and identity crisis, Anthropic researchers wrote: “This kind of behavior would have the potential to be distressing to the customers and coworkers of an AI agent in the real world.”

However, the researchers remained optimistic that some of the mistakes the bot made could be fixed.

“Many of the mistakes Claudius made are very likely the result of the model needing additional scaffolding — that is, more careful prompts, easier-to-use business tools.” – Anthropic spokesperson

The experiment asks whether AI is ready to operate in the workplace without humans. And right, the answer is a pretty clear “no.”