What you need to know
- Microsoft is experimenting with different ways of bringing AI capabilities to Windows.
- Such experiments include bringing OCR to apps like Snipping Tool, Camera, and Photos.
- Microsoft is also toying with the idea of bring generative AI to the Paint app.
This past year has been significant for Microsoft and AI, with almost all new product announcements having some tie-in to an AI-related functionality. Now, even Windows is getting in on the fun with Windows Studio Effects and the upcoming Windows Copilot, announced a few months ago and expected to ship this fall.
But those aren’t the only areas that Microsoft is exploring in adding AI to Windows 11. According to my sources, the company has been toying with bringing AI capabilities to a handful of in-box Windows 11 apps, including Photos, Snipping Tool, and Paint. Some of this functionality will require dedicated hardware, such as an NPU (neural processing unit) or VPU (vision processing unit,) while others may not.
For the Photos app, Microsoft is working on an AI functionality that would allow the app to identify objects or people in photos and enable the ability to cut out and paste those elements elsewhere. This is a functionality that iOS and Android have had for some time, so it’s no surprise to hear that Microsoft is also working to bring it to Windows.
Regarding the Snipping Tool, my sources say the company wants to incorporate OCR (optical character recognition) technology to enable Windows to identify text in screenshots for faster clipboard copying. Microsoft is also working on bringing OCR to the Camera app, allowing users to select text in a photo taken on the device.
Lastly, my sources say Microsoft has also been experimenting with bringing generative AI to the Windows 11 Paint app. Users could ask Paint to create a canvas based on criteria set out by the user, similar to how Bing Image Creator currently works. A new "background removal" feature is also on the cards.
It’s unclear when or if Microsoft will soon ship these AI integrations on Windows 11. My sources say these ideas are still in experimental stages as the company sets out how to incorporate more AI capabilities into Windows. Microsoft is holding an event on September 21, where I’m told it will announce new Surface hardware and further discuss AI in Windows.
I also understand that the next major platform release of Windows, coming in 2024, is expected to incorporate AI more significantly. Right now, the AI capabilities Microsoft is working on for Windows 11 are somewhat surface-level. My sources say the next major release will be where the company begin integrating AI experiences more deeply.
I’ll have more to share on that topic soon, but for now, are you excited about Microsoft’s AI plans for Windows 11? Let us know by adding a comment below!
