How (and why) one Windows Mixed Reality hater became a believer

For the people who follow my work and my social media posts, you might know me as someone who persistently hates on virtual reality (VR).

Part of my skepticism has been driven by Microsoft's handling of the mobile paradigm, which it famously failed at, burning what little good about it the company had down to the ground. I believe that includes developer relationships, faith in the Universal Windows Platform (UWP), and perhaps most importantly, the destruction of trust among its most hardcore fans, customers who might have forgiven the awkwardness of the first generation of consumer-facing Windows Mixed Reality devices.

Of course, whether or not Windows Mixed Reality becomes a success (and VR in general) remains to be seen. But after extensive time with HoloLens, HP's head mounted display (HMD), and Microsoft's new Motion Controllers, I'm starting to believe.

My experience with Windows Mixed Reality

Windows Mixed Reality refers to the entire continuum of VR and augmented reality (AR) experiences on the Windows platform, whether it's AR via a holographic headset like HoloLens, or VR in a fully immersive headset like those offered by Lenovo, HP, and Samsung.

In the near-term, head-mounted displays (HMDs) aren't going to replace laptops or TVs, but they enable compelling experiences that simply aren't possible in other paradigms.

Utilizing "Cliff's House," Windows Mixed Reality's staging area, I was able to view my Movies & TV library on a huge cinema-size screen in a virtual theatre. I was able to snap UWP games, such as GWENT: A Witcher Card Game, side-by-side with other programs, such as GroupMe, Skype, and Microsoft Edge, taking advantage of the spacious environment for a next-level multitasking experience.

I had experienced these things before using Microsoft's HoloLens developer kit, which allows you to position UWP apps as holograms in your existing world. HoloLens has a slightly debilitating letterboxing effect, though, meaning you're fairly restricted when it comes to true augmented immersion.

Instead of simply looking to the left, HoloLens demands that you tilt your entire head to locate the missing holograms. With a Mixed Reality VR HMD, this problem is gone, as the lenses provide full viewing angles.

There are plenty of WMR apps such as HoloTour that are "cool," but I doubt I'll use them more than once. I had my WMR "a ha moment" playing Arizona Sunshine, which is a zombie killin' shoot 'em up from Vertigo Games. Even within the game world, I was still receiving notifications from UWP Twitter and GroupMe, and I even received Xbox achievements for playing the game. The pervasive nature of Windows Mixed Reality really brings the "operating system" experience with you in Mixed Reality, reducing the isolated feeling I had while playing PlayStation VR, which not only blinds you to the outside world, but also stops you from being able to multi-task without removing the headset.

Multitasking in WMR was is simple as pressing the Start button, just like on a regular Windows PC. Arizona Sunshine UWP went into a suspended mode, allowing me to get back into Cliff's House. When I was finished replying to messages, I jumped straight back into the game, right from where I left off.

Windows Mixed Reality still has problems

Some of the issues I encountered aren't so much problems with Mixed Reality as a virtual OS, but more limitations from today's technology.

WMR is still a fledgling idea, and there are clear usability issues that prevent it from achieving mainstream adoption beyond niche use cases. While HoloLens might be great for specific tasks, such as remote learning, adding rich information to an existing scene, and viewing complex 3D models rather than expensive real-world prototypes, the letterbox effect I mentioned earlier limits its capabilities as a true PC replacement. Microsoft is undoubtedly working to rectify this issue with HoloLens, which hasn't yet seen a hardware update since its initial reveal.

VR-based Windows Mixed Reality headsets eliminate the letterbox effect, but there are other annoyances that put a damper on the experience. While it's awesome being able to have tons of screen space for different windows, not being able to see keyboards and mice is a little annoying for orientation. The motion controller keyboard "shooting" isn't anywhere near as fast or intuitive as touch typing, and I still find voice recognition typing to be clunky, at least with my West Midlands British accent. The next step for WMR is sure to bring your physical desk into the virtual space, too, to take advantage of all that multi-tasking real estate, without limiting your ability to actually interact and input information.

Beyond basic usability issues, there are other systemic problems preventing WMR from being a truly comprehensive or intuitive experience. Not only do you need a powerful computer to utilize it, costing $1,000 or more, you also need to pick up a headset that costs at least $350 with the motion controllers bundled. That's a pretty big ask for anyone outside the early adopter arena, particularly since it's arguably less productive for touch typists and Windows power users versus simply using a regular laptop with plain old regular reality.

It's also just clunky, with annoying additional cables, batteries for the motion controllers, giving up your hands for motion controllers, and having to wear something on your head. With HP's headset, I found myself struggling with condensation on the lenses, constantly. All of these things make Windows Mixed Reality feel less intuitive to me than just sitting on the sofa with a laptop. It's kind of cool having a full-size cinema screen for Microsoft Movies & TV, but I'm not sure I'm willing to put up with wearing a headset and having to use motion controllers to navigate it.

What about gaming?

Gaming is a different beast, because the Mixed Reality experience there isn't directly comparable to regular games. It's a completely different, immersive experience, and strangely, my time with Arizona Sunshine took me back to the '90s, in a good way.

When I was as kid, there were arcade machines in our local movie theatre called Time Cop and House of the Dead, which had advanced graphics, and a super accurate physical light guns. They were on-rails shooters, with PlayStation 2-level graphics at best, but they offered a different dimension of immersion that mouse or joystick-bound shooters couldn't. There's something visceral about pointing a physical gun, and VR takes that experience to the next level.

In Arizona Sunshine, you're not just pointing the gun. You're able to manipulate doors as you would in real life, pick up objects and throw them, and traverse the environment at will, as opposed to being stuck on rails. Arizona Sunshine might not be the prettiest game in the world, with dated-looking animations and environments, but the sheer potential is clear.

There are only a handful of decent UWP-based VR games as of writing, with dozens more as part of the SteamVR bridge, but it's exciting to think what the future could hold for this paradigm.

An uphill battle

Microsoft is faced with stiff competition from Apple's mobile-facing ARKit when it comes to building HoloLens-like AR apps, with competition from other gaming storefronts when it comes to support for Microsoft's own Windows Store.

With Windows 10 Mobile shut down, support for native Microsoft Store apps seems to have cratered along with it, with engagement on PC being far lower than that of a phone. Apps ported to the store via Win32 or Electron don't work properly in Windows Mixed Reality, opening instead in a dedicated "Desktop" view.

VR is a paradigm where native apps could function better than desktop PC websites, designed primarily for mice, but I'm not sure how willing developers will be to jump on this ship, considering the install base is going to be miniscule even when compared to that of Windows 10 Mobile. Progressive web apps could help fill the gap, but again, it could be too late for Microsoft, which already seems to be lagging behind iOS and Android when it comes to defining AR app support.

The path to proliferation is going to be a difficult one for Microsoft, but the company seems to understand what it needs to do to ensure it's not left out of yet another computing ecosystem.

I'm no longer a Windows Mixed Reality sceptic, but Microsoft may have a hard job convincing other consumers, and more importantly, developers that it is the right company to back and to usher in this future.