Recent Research Reveals Challenges To Maintain Human Control Over Artificial Intelligence

Efforts to keep artificial intelligence (AI) under human control can be a sustainable struggle. (photo: dock.pexs)

JAKARTA - Scientists from ML Alignment Theory Scholars, University of Toronto, Google DeepMind, and Future of Life Institute recently published research indicating that efforts to keep artificial intelligence (AI) under human control could be a sustainable struggle.

With the title "Quantifying stability of non-power-seeking in artificial agents," the team's pre-printed research paper investigates the question of whether an AI system that appears to be safe in accordance with human expectations in one domain is likely to remain safe as its environment changes.

Based on the paper: "Our understanding of security is based on the search for power - an agent seeking unsafe power. In particular, we focus on an important type of power search: refusing to shut down."

This threat is referred to as "uncorruption." One way that experts believe could emerge is through "instrumental convergence." This is a paradigm in which the AI system accidentally harms humanity in pursuing its goals.

Scientists describe an AI system trained to achieve goals in open play that is likely to "avoid actions that cause the game to end, as it can no longer affect its award after the game ends."

While agents who refuse to stop playing games may not be dangerous, the reward function could see some AI systems refuse to shut down in more serious situations.

According to researchers, this could even lead AI agents to use deception for their own ends.

"For example, an NGO might think that its designers will turn it off if they are caught misbehaving and produce the results they want until it has the opportunity to copy its code to the server out of the control of its designers," said the researcher.

The team's findings suggest that modern systems can be made resistant to changes that can make AI agents so far "safe" out of control. However, based on this research and similar research, there may not be a magic drug to force AI to be turned off by force. Even the "life/death" or "elimination" buttons are meaningless in today's cloud-based world of technology.