What you need to know
- Microsoft is rolling out Visual Search in Bing Chat to the desktop and Bing mobile app.
- The feature allows you to take an image or upload a photo and have the AI-powered chatbot interpret it and ask questions about it.
- Microsoft started testing Visual Search in Bing Chat with a small set of users last month, but is expanding the availability of it gradually.
- Visual Search in Bing Chat will also make its way to Bing Chat Enterprise in the future.
Bing Chat burst onto the scenes this year. The AI-powered chatbot filled headlines and turned heads, especially as OpenAI's ChatGPT tech, which powers Bing Chat, earned a reputation. Following the initial rollout of Bing Chat, Microsoft has added new features at a steady pace. One such addition is Visual Search.
With Visual Search in Bing Chat, users can upload images or take a photo and have the tool scan the photo. Bing Chat can then interpret it and answer questions about the structure. For example, you could take a picture of a building while touring a city and learn historical facts about it. You can also use the feature to take an image of food you have and ask Bing to give you a recipe that utilizes the ingredients.
Visual Search isn't a newly announced feature. Microsoft mentioned it earlier in the year and started testing it with a small group of people last month. But at Microsoft Inspire, the company shared more details on the feature and highlighted what it can do.
Last month, Microsoft's Mikhail Parakhin said that Microsoft had flighted Visual Search in Bing Chat to 5% of users.
Visual Search in Bing Chat is rolling out to the desktop as well as the Bing mobile app right now. Microsoft also plans to bring it to Bing Chat Enterprise "over time," though the company did not specify its plans.
Windows Central take
I'm part of the lucky 5% that has access to Visual Search in Bing Chat at the moment. I played around with it a bit and was surprised by how it "thinks." I shared an image of my American football team and asked the chatbot what the photo showed. Bing Chat correctly identified that the image was of an American football game but incorrectly said it appeared as if the team in white uniforms had the ball.
I scoffed at first, noting that Bing Chat couldn't recognize that a player in a burgundy jersey is clearly holding the ball. But when I corrected Bing Chat, I got a glimpse of the thought process of the chatbot. Bing apologized and stated it "got confused by the referee’s signal." The referee is pointing in a direction that could suggest the team in white has the ball.
While Bing Chat did not get everything about this image correct, it's interesting to see what elements of the photo Bing focused on. Like most AI, I assume Visual Search in Bing Chat will get better over time.
