Microsoft's purchase of Nokia's smartphone business in 2014 added the industry's leading mobile camera technology to its portfolio of assets. Pre-Microsoft, the Nokia 808 PureView smartphone with a 41 MP camera showcased the integration of quality imaging technology with a mobile device.
Nokia's innovative aptitude was evident in both its development of hardware and software critical to mobile imaging tech. This was later complemented with imaging-focused Lumia (formerly Nokia) exclusive apps, after Nokia embraced Windows phones.
For example, augmented reality (AR) via Nokia City Lens was part of Microsoft's ecosystem in 2012, years before AR became "the next big thing." What impact might Microsoft's giving up on smartphones, and by default, its mobile imaging hardware and software investments, have on its AI and mixed reality strategy?
- What are AI, machine learning and digital assistants?
- When Microsoft's AI-driven camera and mixed reality missions meet...
'Cameras as a platform'
Microsoft's previous commitment to "cameras as a platform" (an evolving UI and platform for various apps) was evident in its Camera Lens concept. Users could interact with the world via different imaging apps that were invoked from the camera. The Lumia 1020, with its 41 MP camera and a UI with granular manual controls, also demonstrated Microsoft's commitment to mobile imaging technology. Microsoft introduced hardware Optical Imaging Stabilization (OIS), access to RAW images, and other leading imaging tech long before major competitors.
Nokia City Lens (above video) used the Windows phone camera to search the physical world and overlaid information about surrounding businesses and places of interest on-screen as Microsoft's early steps into AR. Nokia JobLens performed a similar function to reveal potential places of employment for job seekers. This was years before Pokémon Go introduced smartphone-based AR to the masses in 2016.
AI on smartphone cameras is gaining experience that will benefit advanced AR.
Nokia's substantial investments in mobile imaging technology seem to indicate an early realization that smartphone cameras were an evolving platform in themselves. Google and Apple, with a range of camera-focused investments including AI-supported object recognition, facial recognition and Animojis, AR, more sophisticated camera hardware and more, have come to this realization.
Sadly, Microsoft, which had the necessary assets and talent, didn't seem to recognize or value the evolving smartphone camera as a platform. When it cut smartphones it threw away the baby with the bath water.
AR and camera-based AI
To Microsoft's dismay, billions of smartphone users are engaging in an evolving camera-based smartphone experience that transcends merely taking pictures. With integrated AI, built-in features and first- and third-party apps, smartphone cameras are recognizing people, surroundings, creating AR experiences, and much more. Google developers recently created an app that uses AI and a phone's front-facing camera to alert a user that someone is reading over their shoulders.
AI on smartphone cameras is "experiencing" a diverse range of scenarios. This data will ultimately benefit environment, activity and behavior recognition and understanding for implementation in advanced AR experiences. Data is key to machine learning and AI development. As consumers engage AI on iPhone and Android phone camera platforms, Apple and Google are acquiring vital information to evolve their AI and camera platforms.
Microsoft kept Windows 10 Mobile alive to continue development of ARM and cellular technology for future implementation in project "Andromeda." Perhaps continued investments in camera-focused smartphones (for a niche market) over the years would have allowed ongoing development of Microsoft's camera as a platform.
Though it wouldn't have had Apple's and Google's market presence, the Windows phone user base would have been a valuable resource to contribute feedback to Microsoft's camera-as-a-platform efforts (even if operating at a temporary financial loss). This data would, in turn, be transferred to the company's leading mixed reality and AI efforts.
The cost of cutting smartphones
Microsoft, Apple, Google and other companies see AR as personal computing's future. AR hardware and software must, therefore, be able to accurately perceive and understand its environment. Quality optical hardware, software and AI are fundamental to AR's evolution. Thus, smartphone-focused cameras as a platform is an evolving component that adds value to a company's AR efforts.
Microsoft's cutting smartphone resulted in its cutting Nokia talent responsible for innovative camera tech. Absence from the smartphone space also erased what little mindshare Windows phones achieved as a result of its camera prowess. If AR is the future of computing, mobile camera technology is key.
Ironically, the recent publishing of a patent filed in 2016 by ex-Nokia employee and lead inventor Eero Tuulos reveals Microsoft's attempts to solve for the positioning of a camera on a folding mobile device. Indeed, if Microsoft had kept marketing camera-focused smartphones, the continued investments in mobile imaging technology may have beneficially impacted the implementation of this patents idea in the folding mobile device we believe Microsoft is working on.
A missed opportunity ...
Microsoft has demonstrated powerful AI-driven camera technology (using existing camera systems) that can recognize people, places, and activity and proactively act on what it sees. The merging of that level of AI with its mixed reality efforts is likely inevitable.
Still, the evolving of miniature mobile camera tech and software, the data gleaned from AI-human interactions on mobile camera platforms and the mindshare and goodwill even a niche camera-focused phone would have provided for application in wearable AR tech is likely a missed opportunity for Microsoft.
Related reading
