Meta Finds Content Use That AI May Make In Online Operations

Meta found content on Facebook "possibly generated by artificial intelligence (AI)" (photo: dock. Facebook)

JAKARTA - Meta announced on Wednesday May 29 that it had discovered content "possibly produced by artificial intelligence (AI)" that was used misleadingly on Facebook and Instagram platforms. The content includes comments praising Israel's handling of the war in Gaza published under posts from global news organizations and members of the United States congress.

The social media company mentioned in their quarterly security report that the accounts disguised themselves as Jewish students, African Americans, and other concerned citizens, targeting audiences in the United States and Canada. Meta blamed the campaign on Tel Aviv-based political marketing company STOIC.

STOIC itself did not immediately respond to the allegations.

Although Meta has discovered fundamental profile photos produced by artificial intelligence in influence operations since 2019, this report is the first time to unveil the use of AI technology produced by text since its appearance in late 2022. Researchers have been concerned that AI is generative, which can quickly and cheaply generate human-like text, images, and audio, could lead to more effective disinformation campaigns and affect election results.

In a press release, Meta security executives said they removed the Israeli campaign quickly and did not think that the new AI technology was hampering their ability to disrupt the influence network, which was a coordinated effort to encourage certain messages.

"There are several examples across the network about how they use AI-generative tools that are most likely to create content. It might give them the ability to do it faster or with more volume. But it doesn't really affect our ability to detect them," said Meta's head of threat investigation Mike Dvilyanski.

The report highlights six hidden influence operations that disrupted Meta in the first quarter. In addition to the STOIC network, Meta also shut down Iran-based networks focused on the Israeli-Hamas conflict, although it did not identify the use of a generating AI in the campaign.

Meta and other tech giants have struggled to tackle the potential abuse of new AI technology, especially in elections.

The discovery of content that is likely produced by AI, which is used misleadingly, highlights the challenges faced by social media companies in detecting and reducing such activity. It also confirms the need for continuous vigilance and adaptation of technology by social media platforms to counter online-evolving manipulation and disinformation tactics.