Pinterest Leverages AI To Improve Users' Shopping Experience

Pinterest, now provides an even more enjoyable shopping experience. (photo; Charles Deluvio / Unsplash)

JAKARTA - Artificial intelligence and augmented reality can work together to support innovations that can change the shopping and advertising game.

"Pinterest combines artificial intelligence with augmented reality and combines diversity, as well as inclusion in a way that helps the company profit," said Jeremy King, chief engineering officer at Pinterest, Jeremy King, in an interview with Venturebeat Founder and CEO Matt Marshall at the Transform 2021 virtual conference, Tuesday 27 July.

Seeing more than 80 million people searching for beauty items every month on the app, Pinterest had to innovate how people can see themselves in the products they are interested in.

Therefore, the company created the Try On tool, which allows users to use their phones to see how they would look with different types of makeup chosen.

“AR Try On is our augmented reality feature that lets you use the lens camera on your phone to try lipstick and eyeshadows and filter makeup,” said King.

"You just have to pick up your phone, and point it right at your face and you can do a search for something like red lipstick or makeup ideas and you'll see not only a palette of skin tones but also buttons to try on."

Pinterest works to prevent bias in the data by diversifying data sources and testing computer vision and machine learning algorithms across a wide range of skin tones and makeup. This tool is capable of understanding and processing the range of skin tones in low lighting scenarios.

“Building diversity and representation is not just the right thing to do. For us, this is great for our business,” said King.

Then the result, when using the Try On feature with AR, customers are five times more likely to buy the product. Pinterest has worked to improve the customer and vendor experience in other ways too, turning the value of diversity and inclusion into action.

“We frankly heard from pinners that we were naturally irrelevant or mixed in our recommendations, and they should add more descriptors to their search to filter them out,” said King.

"We thought of it as an action item to create this capability not only to allow more diverse content to emerge, but to integrate new features like AR Try on. And once we got it, we really got the engagement we were looking for," he added.

The company has also worked with content creators to produce content while also working with retail partners, to ingest hundreds of thousands of products from around the world in the Pinterest catalog.

“Creators come to create unique content. We want to be able to use not only creator content, but also our computer vision technology, to identify every image and every one of those billions of times so you can create truly great experiences."