My AI Chatbot Gives Inappropriate Response, Snapchat Launches OpenAI-Based New Tool

My AI is a chatbot and not a true friend. (photo: dock. snapchat)

JAKARTA - More than a month ago Snapchat's Artificial Intelligence (AI) chatbot was launched. However, the company is concerned about some potential abuse. Now, a new tool is here to make it safer.

"As part of our collaboration to improve My AI, we want to share updates on some of the security improvements we've recently made as a result of our learning, along with the new tools we plan to implement," Snap said in its official blog, quoted Thursday, April 6.

The tool comes in response to a Washington Post report highlighting the bot responding in an unsafe and inappropriate manner.

The social media giant said after the launch, many people were trying to trick chatbots into giving responses that were not in accordance with their guidelines.

To keep AI responses, Snap has entered a new age filter, where AI can find out the user's date of birth and give them age-appropriate responses. Although Snapchatter (as Snap users) never told My AI about their age in conversations.

The company stated that chatbots will consistently consider their age when involved in conversations with users.

Before Snapchatter is allowed to use My AI, the company will show them orientational messages that clarify all messages with My AI will be saved unless they delete them.

"However, because MyAI is a chatbot and not a true friend, we deliberately treat related data differently, because we can use conversation history to continue to make My AI more enjoyable, useful, and safer," Snap said.

The inappropriate responses include referrals to violence, explicit sexual terms, the use of illegal drugs, sexual abuse of children, intimidation, hate speech, insulting or biased statements, racism, misogini, or marginalizing underrepresented groups.

For this reason, Snap has also added OpenAI moderation technology to My AI, aiming to assess the potentially dangerous severity of content. Now, the company will temporarily block access to AI bots for users who abuse the service.

"We continue to encourage Snapchatter to use in-app reporting tools if they receive any responses from My AI and send us feedback on their entire experience with the product," Snap said.

Snap also plans to provide more insight in the coming weeks to parents or guardians about children's interaction with bots at the Family Center, which launched last August.

Parents will be able to use the Family Center to see if their child communicates with My AI, and how often. Both guardians and teenagers must participate in using the Family Center to use this parental control feature.