How Minecraft Earth went from HoloLens demo to Pokemon GO rival

Minecraft Earth renders the blocky bestseller a reality, pitching an ambitious augmented world bound for iOS and Android. Collecting resources and scaling 3D creations, Microsoft's upcoming free-to-play is poised to emulate Pokemon GO's successes, drawing players to the streets across the globe. And with the public "early access" debut around the corner, we sat down with Minecraft Earth Executive Producer Jesse Merriam, talking the title from concept to execution.

Minecraft Earth comes at an ideal time for Microsoft

With Minecraft Earth's worldwide launch looming, Pokemon GO comparisons are evident, embracing the same underlying setup as the 2016 mobile hit. Niantic's fusion of augmented reality (AR) and location data blossomed to an unseen scale, marking a significant evolution for on-the-go experiences. Riding on the series alone was bound for success, but real-world ties left a cultural impact still unmatched by would-be successors.

It's undeniable that Mojang boldly follows in Niantic's footsteps, embracing a similar combination of AR and mapping that gave Pokemon GO its charm. But the studio looks to set itself apartment with fundamentally deeper integration, constructing an experience focused on the quirks of AR, rather location-dependent features.

"I don't know that there's a large-scale augmented reality game in use yet. There's off switches in all of them, and they're mostly used that way. We are an augmented reality game, in that we embrace the play space of the real world. But the comparisons really stop there. Because what you do in this game is not like what you do anywhere else," said Merriam. "I think we stand out in that we are augmented reality-first and an augmented reality-only game. That allows you to have that experience and it's the only way to play."

Early virtual and augmented experiences seek to prove the capabilities of the technology but often falter with long-term engagement. Staying power was fundamental to the initial Minecraft Earth pitch, advocating open-ended gameplay and replayability, already synonymous with Minecraft's sandbox. "[W]hatever experience we create should reasonably be something you'd want to do for an hour. I think with a lot of augmented reality and interactive gaming, it's a really hard creative endeavor to get past that. Those five minutes."

While the Pokemon GO boom drew industry-wide attention, Minecraft Earth's AR prowess hails back to its initial HoloLens demonstrations from early 2015. The ambitious self-contained headset is among pioneers of holographic computing, revealed with a glimpse of tabletop Minecraft, layered upon the real world. The then-newly-acquired series never saw a formal HoloLens debut, but it's clear where learnings translated to the Minecraft Earth.

"Previously, I had created the HoloLens version and the VR version as well. My job prior to being Executive Producer for Minecraft Earth was Executive Producer for Minecraft, for the five years before that. And as a function of expanding platforms, when we got to augmented reality, it was really visceral and real. And as you can probably relate to, when we were kids, we all want to jump into our favorite movie and our favorite video game."

"I remember when I first had the HoloLens on, it was a prototype, the whole studio only had one. I looked up, there was a sheep, I moved toward the sheep, and it looked at me," said Merriam. "I tried to pet it and I start chasing it down the hall – my dev manager is following after me, trying to keep me from breaking our one HoloLens. But I was in it, it was real. There's something really connecting about learning that a sheep can come up to here on you, it suddenly becomes so much more real."

The HoloLens demo never made it beyond an E3 2015 presence, as Microsoft doubled down on enterprise implementation. The second-generation HoloLens lies around the corner; but still without entertainment applications or a consumer-friendly variant. Minecraft Earth is the new AR focus, but Mojang is yet to rule out HoloLens entirely.

"That is an exciting experience. HoloLens is yet to have a consumer release, and so when it does, we will endeavor to bring something exciting to it. That was always an experiment to create a consumer game on the HoloLens. When I joined Minecraft, Pocket Edition supported iOS and Android, and over the years, as we got to Better Together [Update], we had expanded into that one codebase addressing 17 platforms. So HoloLens to us was just another great platform. We worked on it, we went from there to VR, and then a number of other things."

Unlike current mobile rivals, Merriam discussed an effort to avoid "off switches," pitching AR at the heart of Minecraft Earth. The developer appears committed to this philosophy, serving all building and play exclusively via the camera lens. When questioned on Microsoft's continued push for accessibility, Merriam did discuss ongoing research for alternative inputs, but remains averse to stripping out AR.

"[T]here are some considerations for pinch and zoom and rotating the build. That we're considering because there are different play styles and accessibility reasons we want to accommodate. Over time I think you'll find that build plates will probably have configurations like that."

Minecraft Earth strives to seamlessly blend content into surroundings, with positioning a fundamental role in immersion. This means challenges with pinch and zoom, finding balance in the name of accessibility. "If we optimized just for [four players], we don't, because all four of the people should be at the same vantage point. When you see somebody's phone, how far away it is, you should know where they're building. You should know what they're doing. And if they can zoom in, it changes the perspective for people. […] It's the same reason [that] we don't allow you to go up or down."

Taking Minecraft global with Azure's helping hand

Minecraft Earth soon delivers its augmented world to the masses, leveraging an array of supporting technologies to realize that vision. Mojang has broadened scope parallel to industry advancements, spanning mobile AR, cloud computing, and data accessibility. "There's a conglomerate of complicated technology coming together in this experience that had to evolve," said Merriam. "We had to wait until cloud anchors could be a thing and working with the cognition team to create them. ARKit and ARCore, and the speed of phones and OpenStreetMap to give us contextual data about this whole play space of a planet."

The spotlight may fall on Minecraft Earth's iOS and Android AR achievements, but sophisticated underlying technologies are in play too. The demanded scale and accuracy of geographic gameplay requires Redmond's cloud empire, tapping into recent strengths of the Microsoft Azure portfolio. Cognitive computing and Azure Spatial Anchors were among essential tools, essentially poised for creating a digital twin of our world, driving precision within just three centimeters.

"That's a fantastic service and a great partnership within Microsoft, where they are building a web of the real world, a mesh if you will, of point clouds. As we're scanning for an anchor, we're uploading the shape of the world around us – in triangles, no photos are being uploaded – so that we create an anchor, so that when multiple people come in, everyone's anchoring in the same spot on the park and everybody sees the exact same place."

"The web of data centers that Azure has today we're pretty confident in. I will be flying to Africa soon, just to validate that! I've done a lot of time in Tokyo recently and I want to stand in Africa, I want to stand in Australia, just make sure those spots that everybody tells me are just as good, feel just as good. But yes, all indications so far is that we think that this scales globally in a very responsive way."

Microsoft kicked off the launch with closed beta testing, trialing its services among a limited pool of applicants. The pre-release client served a shell of the Minecraft Earth experience, stripping gameplay to simple resource collection and building, but proved valuable long-term deployment. Considerable interest unsurprisingly followed beta registration, only attested by thousands with unauthorized Android copies since. "We have a tremendous number of passionate players," said Merriam, reflecting on its beta testing period. "We're getting good experiences and collecting and building and learning. These adventures we're showing you today really complete the loop. [W]e're excited about that initial findings though were incredibly positive. And it turns out our services have been rock solid."

Minecraft Earth's location-based gameplay fostered a hyper-focused rollout, first limiting tests to iOS devices across five pre-selected hotspots; Seattle, London, Stockholm, Tokyo, and Mexico City. While it later expanded to Android, tight criteria left many applicants excluded.

"We wanted to test multiplayer, and we wanted to have some sort of density. That's actually pretty hard to do, especially. The beta program for Apple maxes out at 10,000 and on Android maxes out at 400,000. So we had to figure out how to find a space where we can get density to observe multiplayer experiences, rather than just have single-player experiences all over. That's why we had a very uncommon deployment pattern that was really targeted. Picked that, just trying to look at the density of play styles, so we could imagine there's a little chunk of the earth that's going to operate the way early access would."

As for how Mojang selected its beta cities, Merriam details a drive to understand the challenges of deployment. "[If] you look at placement algorithms, we're placing adventures in the park. We want to make sure we understand the globe. We tried to pick the different spots in the globe," said Merriam. The geographic diversity of these cities brought new hurdles to overcome before full global mapping. "In Mexico City, is it different than London? Is it different than Tokyo? Because we know how to place things in Seattle, we're really good at that. We wanted to learn the rest of the world. So we tried to take a swathe of places that we thought honestly would be the most complicated."

The Minecraft Earth beta shortly winds down, as Microsoft heads for an Early Access debut, rolling out on a per-country basis. As for where Minecraft Earth hits next? Mojang isn't committed to a formal roadmap, looking to adapt deployment on the move.

"We're not going to share it. Probably for the biggest reason is I keep rearranging it. We're going to optimize that for learning. Because what we're doing is populating the planet and figuring out the right places to put experiences. Testing our services and making sure our services work. Testing our latency to various endpoints. We're going to roll out somewhere and pick a country. We'll probably start with some small countries, look at the results, and whichever thing goes wobbly, we'll probably pick another country similar."

Minecraft Earth is slated for an early access debut this October, providing the first draft of Mojang's new AR spinoff. The free-to-play iOS and Android apps take resource collection, crafting, and building global, alongside new curated Adventures primed for multiplayer action. Kicking off with Seattle, London, Stockholm, Tokyo, and Mexico City, expect new additions over the months ahead, before universal availability in late 2019 – excluding China among other blacklisted regions.