Palo Alto Networks Reveals the Dangers of Dark LLMs, AI as a New Weapon of Cybercrime

JAKARTA - The biggest risk of artificial intelligence (AI) today lies in its dual purpose, both for positive and negative purposes such as automating phishing, fraud, and malware on a large scale.

Large language models (LLMs) such as ChatGPT and Google Gemini have become part of the daily activities of many organizations and communities.

However, the latest research from Palo Alto Networks Unit 42 shows that the emergence of malicious or dark LLMs has the potential to significantly change the cybercrime landscape, including in Indonesia.

In a report titled The Dual-Use Dilemma of AI: Malicious LLMs, Unit 42 revealed the existence of models such as WormGPT, FraudGPT, and KawaiiGPT that were built without security systems and are now openly traded through Telegram and dark web forums.

This threat is considered increasingly urgent for Indonesia, given the high dependence of the community on instant messaging applications, e-commerce platforms, and digital public services.

Unit 42 revealed three main impacts of dark LLMs on the cybercrime landscape in Indonesia.

First, the ability of AI to generate highly precise language allows perpetrators to create phishing messages and business email infiltrations that appear very convincing, imitating the communication style of company leaders, financial institutions, or government agencies, so as to be able to exploit the victim's trust.

Second, this technology encourages the commercialization of cybercrime with the ability to instantly generate malware, phishing kits, and data theft scripts, which previously could only be done by perpetrators with high technical expertise.

Third, with the removal of technical barriers, cybercrime is becoming increasingly democratized, allowing low-skilled perpetrators to carry out digital fraud and extortion quickly, turning cybercrime into a cheap, repetitive, and massive operation.

Therefore, Palo Alto Networks emphasizes the importance of establishing standards and frameworks that regulate the proliferation of harmful AI models as well as requiring the implementation of best security practices for policymakers.

As Indonesia draws up a national AI roadmap, Unit 42 emphasizes that the main challenge is not to restrict or ban the use of AI tools, but to build cybersecurity resilience against fast-paced and large-scale AI-based attacks.

"Collaboration between the government, regulators, and the private sector is needed to ensure that safe AI practices are embedded in the AI roadmap and national governance framework," he said.