Researchers at the University of Chicago Create Tool to Poison AI

The University of Chicago developed a tool that provides artists with the ability to "made poison" AI. (photo: dock. pexels)

JAKARTA - Researchers at the University of Chicago have developed a tool that gives artists the ability to "poison" their digital artwork. This was done to stop developers from training artificial intelligence (AI) systems with their work.

Called “Nightshade,” named after a family of poisonous plants — the tool modifies images in such a way that its presence pollutes the data sets used to train the AI ​​with incorrect information.

According to a report from MIT's Technology Review, Nightshade alters the pixels of digital images in order to trick AI systems into misinterpreting them. For example, Tech Review mentions that this tool can convince AI that a picture of a cat is actually a dog and vice versa.

In doing so, the AI's ability to produce accurate and theoretically reasonable output will be compromised. In the example above, if a user requests an image of a “cat” from a tainted AI, they might instead get an image of a dog tagged as a cat or a composite of all the “cats” in the AI's training set, including images of dogs that have actually been modified by Nightshade tool.

An expert who looked at the work, Vitaly Shmaticov, a professor at Cornell University, argued that researchers "don't yet know a robust defense against these attacks" — with the implication that robust models like OpenAI's ChatGPT could also be at risk.

The research team behind Nightshade is led by Ben Zhao, a professor at the University of Chicago. This new tool is actually an extension of their existing artist protection software, called Glaze. In previous work, they devised a method by which an artist could obscure, or "glaze," the style of their artwork.

An artist creating a charcoal portrait, for example, could have it coated with a glaze to make it appear to an AI system as modern art.

According to Technology Review, Nightshade will eventually be implemented into Glaze, which is currently available for free for web use or download.