New York City Mayor, Defending Misinformation AI Chatbot

New York City Mayor Eric Adams maintains the use of artificial intelligence chatbots (AI) despite giving incorrect answers. (photo: x @NYCMayor)

JAKARTA - New York City Mayor Eric Adams maintains the use of the city's new artificial intelligence chatbot (AI) which has recently been trapped in giving wrong answers or providing advice that, if followed, will violate the law.

When launched as a trial in October 2023, MyCity chatbots were promoted as the use of the first AI technology across the city, something that would provide business owners with "information that action and trust can take" in response to questions typed into online portals.

This is not always proven: journalists at the investigation outlet The Markup first reported last week that the chatbot provided misinformation. Chabot mistakenly suggested that employers could take part in their workers' tips, and that there were no regulations demanding the boss notification of changes to employee schedules.

"There are some things that go wrong, and we have to fix them," Adams, a Democrat politician, told reporters on Tuesday, April 2. He stressed that it was a trial program. "Every time you use technology, you need to put it in a real environment to solve the problems," he added.

Adams has been a passionate supporter for using untested technology in the city with optimism that hasn't always proven to be true. He placed a 400-pound robot with a cryptic ovoid shape at Times Square subway station last year which he hopes will help police prevent crime. But the robot retired about five months later, where road users noted that the robot was never seen doing anything. The robot also cannot use stairs.

While the chatbot that has been complained about so far remains online. Last Thursday still sometimes gave the wrong answer. He also said shop owners are free not to accept cash payments, apparently not aware of the 2020 city law that prohibits shops from accepting cash payments. Chatbot also still thinks the city's minimum wage is 15 US dollars per hour, although it is raised to 16 US dollars starting in 2024.

Chatbot, which relies on Microsoft Azure AI services, appears to be being tricked by problems common to the generative AI technology platform called ChatGPT, which is sometimes known to make something untrue or state lies. Neither Microsoft nor City Hall responded to questions about what caused the error and how to fix it.

The City has updated warnings on the MyCity chatbot website, noting that "the response can sometimes be inaccurate or incomplete" and telling businesses not to use their responses as legal or professional advice."

Andrew Rigie, who defended thousands of restaurant owners as director of the NYC Hospitality Alliance, said he had heard from business owners who were confused by the chatbot's response. "I appreciate the city's efforts to use AI to help business, but it has to work," he said.

Rigie also warned that following some chatbot guidelines could have serious legal consequences. "When I ask and then I have to go back to my lawyer to find out if the answer is correct, it beats the goal," he said.