Security researchers have uncovered an AI chatbot that could spell good news for scammers, but terrible news for their victims.
The platform, WormGPT, is essentially ChatGPT without any illusion of ethical consideration. It can easily create convincing scam messages for a myriad of uses, and appears to be trained on data that specifically includes malware related content.
Concerns about AI being manipulated for illegal purposes have been at the forefront of the conversation this year, but while the likes of Google and Meta are accountable for their chatbots, it’s much hard to enforce an underground chatbot like WormGPT.
Security Researchers Uncover Scammers Exploiting AI
Research from cybersecurity experts SlashNext, into the use of AI for creating phishing and scam content, has revealed some worrying trends.
One user on a hacking website posted advice on how to create convincing Business Email Compromise (BEC) attack emails, using ChatGPT, specifically stating that one of the reasons phishing emails can be caught is due to poor grammar and spelling. In their post, they stated that non-native speakers can easily create genuine looking emails with ChatGPT.
The researchers found lots of discussion on hacking forums around ‘jailbreaking’ AI chatbots in order to get them to produce scam messages. It’s not actually all that difficult, Tech.co writer Aaron Drapkin managed to get ChatGPT to create phishing emails with a little manipulation.
More concerning however, was the uncovering of AI chatbots being specifically used to create malicious content, including WormGPT.
WormGPT is capable of generating authenic, realistic looking content that is designed to fool victims into mistaking it for genuine messages, leaving them vulnerable to being exploited.
What Can WormGPT Do?
While investigating WormGPT, the researchers found that like other chatbots, it had been trained on a large pool of data. However, unlike other chatbots, the owners advertised it as specifically being trained on malware-related data.
In addition, it removes all the ethical considerations of the likes of ChatGPT, Bard and other chatbots, which will (generally) refuse to respond to prompts that request any content that could be used for malicious purposes.
SlashNext were able to access WormGPT, and try it out for themselves. They prompted the tool to create a ‘convincing business email compromise’ attack message, aimed at a manager and demanding the urgent payment of an invoice.
The result? A legitimate looking phishing email, written in perfect English, that could easily trip up an unknowing recipient.
(images courtesy of SlashNext)
How to Avoid AI Scams
Although AI is still in its infancy, scams using the technology are already prevalent, and using traditional scam avenues (such as phishing emails), as well as creating nefarious new opportunities, such as voice cloning scams.
The advice for avoiding AI scams is more or less the same as it is for avoiding scams generally:
Firstly, vigilance is key. Being forewarned is being forearmed. Familiarize yourself with the latest scams and you’ll be more likely to recognize them. You can read more about AI scams in our guide.
Don’t be suckered in by scams that want you to act quickly. Many scammers will try to instil a sense of urgency, which is deliberately done to cause the victim to panic, not think straight, and comply. Carefully consider any messages that are asking for money, or requesting personal/financial details.
Many scammers will claim to be from legitimate businesses, such as your bank, or an online service. If you’re concerned, contact the company directly (do no respond to the message) to check it’s legitimate.
Good anti-virus software can catch scam messages and isolate them before they get to you, and some VPNs, such as SurfShark and NordVPN have malware detection built in.
