Google's AI Mode Feature Is Getting More Sophisticated, Can Answer Questions Through Just Photos

JAKARTA - Google is expanding the presence of the AI Mode multimodal search feature to millions of Labs users in the US, with registrations already open through the Google (Android and iOS) app.

First launched for Google One AI Premium subscribers in early March, this extended AI-based feature now allows users to search for information simply by uploading images.

By combining Google Lens' visual search capabilities and Gemini's multimodal intelligence, users can take pictures or upload images, ask questions about what they see, then get a complete and contextual answer on Google Search.

In a demo video uploaded by Google on its blog, Google provides an example while photographing a book rack. From there, AI Mode can immediately recognize the titles of the books in it, and provide recommendations for similar books, complete with a purchase link and reviews.

This experience combines Lens' advanced visual search capabilities with Gemini's customized versions, so you can easily ask complicated questions about what you see, "said Product VP, Google Search, Robby Stein in his blog.

Stein added, with Gemini's multimodal capabilities, AI Mode can understand the entire scene in an image, including the context of how objects relate to each other and material, color, shape, and unique arrangement.

The result is a response that is very nuanced and contextually relevant, so you take the next step, "he explained further.

Nonetheless, this feature is still in the experimental stage, and Google continues to collect input from users for further development.

"We also continue to improve our experience and today we present our advanced multimodal capabilities at Lens to AI Mode," Stein concluded.