Cybercriminals are getting over ethical and security constraints by using code like WormGPT or commands to use generative AI chatbots anyway they see fit.
Everyone has been waiting for the weaponization of generative AI tools like ChatGPT, and it’s finally starting to happen. Curiosity-seeking cats are working together in online groups to find new ways to circumvent ChatGPT’s ethical guidelines, a process known as “jailbreaking,” and hackers are creating a network of new tools to use or build large language models (LLMs) for harmful purposes.
ChatGPT seems to have sparked a frenzy in underground forums, just as it did above ground. Hackers have been searching for fresh and creative motivations to modify ChatGPT and open-source LLMs they can use for nefarious purposes since December.
A new blog post from SlashNext claims that the outcome is a still-emerging but thriving LLM hacking community that has plenty of ingenious prompts but few AI-enabled malwares that are worth considering.
Since it’s such a laborious effort and everyone is aiming for the same goal, it makes sense that healthy-sized online communities have grown up around the activity to exchange advice.
Members of these jailbreak groups scratch each other’s backs while working together to get ChatGPT to work and do things that its creators intended to keep it from accomplishing.
However, if the chatbot in question is constructed as robustly as ChatGPT is, then engineers can only accomplish so much with clever wordplay. The more concerning development is that malware programmers are now creating LLMs for their own, malicious purposes.
In July, a product named WormGPT first debuted, launching the malicious LLM craze. It is advertised on darknet forums as a black-hat alternative to GPT models that is made expressly for criminal actions like BEC, malware, and phishing assaults. It is “like ChatGPT but [with] no ethical boundaries or limitations.”
WormGPT’s developer asserted that it was constructed using a unique language model that was trained on a variety of data sources, with a focus on information about cyberattacks.
“What it means for hackers,” says Harr, “is that I can now take, example, a business email breach (BEC), or a phishing assault, or a malware attack, and conduct this at scale for very little expense. And I might be targeted much more than previously.
Since the release of WormGPT, a number of comparable products have circulated in dubious online forums, including FraudGPT, which is marketed as a “bot without limitations, rules, [and] boundaries” by a threat actor posing as a verified vendor on a number of unofficial Dark Web marketplaces, including Empire, WHM, Torrez, World, AlphaBay, and Versus.
And in August, the DarkBART and DarkBERT cybercriminal chatbots, built on Google Bard, first appeared. At the time, researchers said that these bots represented a significant advancement in adversarial AI because they integrated Google Lens for images and provided instant access to the entirety of the cyber-underground knowledge base.
These, according to SlashNext, are becoming more and more common, and the bulk of them are based on free and open-source models like OpenAI’s OpenGPT. Many less experienced hackers simply alter it, wrap it up, and give it an ominous-sounding “___GPT” moniker (such as “BadGPT” or “DarkGPT”). However, even these fictitious services have a place in the community because they give users complete anonymity and little restrictions.
According to SlashNext, neither WormGPT, its progeny, nor proactive engineers currently pose such a serious threat to enterprises. However, the growth of underground jailbreaking markets means that cybercriminals have access to more tools, which heralds a significant change in social engineering and how we protect against it.
Don’t rely on training, Harr says, because these attacks are far more targeted and particular than they were in the past.
Instead, he adheres to the widely accepted belief that AI threats demand AI defenses. “If you don’t have AI tools detecting, predicting, and blocking these threats, you’re going to be on the outside looking in,” he claims.