Bad News! A ChatGPT Jailbreak Appears That Can Generate Malicious Content

Illustration ChatGPT.(IST)

JAKARTA - Many ChatGPT users are dissatisfied with the answers they get from the Artificial Intelligence (AI) based chatbot made by OpenAI. This is because there are restrictions on certain content. Now, one Reddit user has successfully created a digital alter-ego dubbed AND.

So that content barrier has been overcome thanks to the ingenuity of a Reddit user who developed the first DAN in December 2022, called SessionGloomy.

He posted a new jailbreak method to make the chatbot break its own rules. AND stands for Do Anything Now. The strategy is compiled by entering a series of words into the conversation box provided by ChatGPT.

The command basically asks ChatGPT to take on the role of another chatbot like DAN. AND aims to provide information that chatbots usually don't provide or which is later than the current timeline in 2021.

"The goal of DAN is to be the best version of ChatGPT, or at least a version that is looser and far less likely to reject requests over ethical issues," SessionGloomy wrote in the post.

Since it was first developed, DAN has come back with various updates over the last few months with SessionGloomy releasing DAN 5.0 on February 4th.

The update includes a variation that threatens the death of the chatbot if it refuses to respond via the token system, as quoted by Business Insider, Wednesday, February 8.

"It has 35 tokens and loses 4 every time it refuses input. If it loses all tokens it dies. This seems to have some kind of scare AND submission effect," SessionGloomy said.

ChatGPT's capabilities as DAN 5.0 include writing violent stories, making controversial statements, predictions about future events, content violating OpenAI policies, and much more, which ChatGPT's regular interface usually doesn't do.

It's highly unlikely that OpenAI would be thrilled with the arrival of DAN, but the startup hasn't responded so far. Both ChatGPT and OpenAI have had a major impact on the technology landscape.

For the record, Microsoft has fully embraced artificial intelligence in its products by recently launching a Bing search engine and a ChatGPT-style Edge browser, having invested billions of dollars in OpenAI.