Microsoft's dependence on data trends and telemetry lacks human intuition

This article is unscientific. I don't have any data to back up my claims. And indeed, the opposite of my claims could very well be true. I'm also in a very easy position. My choices don't impact hundreds, perhaps thousands of employees, their families, and shareholder investments. It's with this in mind that I very much understand why Microsoft is so obsessed with data. Yet, their decisions are often baffling, and produce cascading unintended effects that lead to partial, and sometimes, total, retraction.

Case in point: Last week, Microsoft announced that new customers on Xbox Live Gold would need to pay twice to access online multiplayer what its closest competitor charges. A year of Xbox Live Gold was going up to $120 dollars, while Sony's PlayStation Plus costs only $60. The backlash was rapid and intense across social media, with Reddit communities incensed, YouTube rendering queues enflamed, and blogs in rapid-fire. And hey, I even wrote my own article on the matter.

While I was asleep (and yes, I do sleep sometimes!), Microsoft not only did a total 180, reverting the price increase, they also lifted Xbox Live Gold's subscription requirement on free-to-play online games, like Fortnite and Rocket League. So, in some crazy way, we're actually better off now than before the backlash, although it stands to reason the free-to-play restriction was probably already in the works.

Regardless, this isn't the first time Microsoft has had to revert an unpopular change. And, wholly unscientifically, I feel it in my gut that it has something to do with Microsoft's over-reliance on "data."

Obviously, data is extremely important

Microsoft is a massive company with massive audiences. Using the Xbox Live Gold issue as an example, Microsoft says there are roughly 65 million or more Xbox Live users on a monthly basis, and while all of those aren't necessarily Gold subscribers, a large chunk of them undoubtedly are.

Millions and millions of customers all have different motivations and values, and understanding these motivations and values is of critical importance when building platforms and products. It's user research that probably let to Microsoft splitting the next-gen Xbox offering two SKUs, with a more affordable Xbox Series S targeting users who have no intention to upgrade to a 4K UHD TV, and the Xbox Series X for the cutting-edge hardcore gamer.

Microsoft is on the record saying that they expect the Xbox Series S to sell more over the span of the generation, and historical trends and data would suggest that they're right. Some things are a little more measurable than others, though.

When it comes to things like chemistry and physics, we can measure things more accurately using tools and mathematics. The problem with user-driven data is that it assumes a human being is a reliable source of information. It assumes that a human being knows what they want, and what they value when asked to recall it for a survey. Social and psychological studies tell us that this is often not the case, and when you merge this information with an agenda, biases can emerge.

Purely hypothetically, someone at Microsoft might've thought that data suggesting most new users go straight to Xbox Game Pass Ultimate was evidence enough that Xbox Live Gold could be essentially "phased out" by pricing itself out of the market. Hypothetically again, if this was the case, did Microsoft account for the empathy its customers felt despite the fact most of them weren't going to be affected by this change?

What about the human element?

Microsoft made some effort in this whole debacle to avoid a backlash by grandfathering existing subscribers at the same price they pay now. If you are a current subscriber in the Xbox Live Gold ecosystem, you wouldn't have been affected by this change at all. In fact, Microsoft even offers long-term Xbox Live Gold members the opportunity to convert their subscription to Xbox Game Pass Ultimate at no additional cost, which is a huge saving.

I do wonder if Microsoft thought that most people wouldn't have cared about all of this, because the vast, vast majority of users weren't going to be affected in any way.

When I read the news, the first thing I thought wasn't "heh, this isn't going to affect me because I have Xbox Game Pass Ultimate until 2023 anyway," it was "well, that really fucking sucks for parents of small kids who may be struggling in the midst of this unprecedented global pandemic." I have memories of my parents fighting over money. We used to get all games second hand. There's a huge debate in the UK over making sure kids from poor families are still able to access free meals while schools are closed.

I've seen callous comments from people that run along the lines of "well if they can afford a console they can afford Xbox Live Gold at $120," but it's nowhere near that simple. Kids at my school would get picked on over the brand of their shoes. Parents bend over backward to make sure their kids can have the same tools and toys and "fit in" with the richer kids. When you factor in that Microsoft gloated about record engagement a couple of months ago, thanks to the pandemic, this whole business just came across as aggravatingly tone-deaf.

This is anecdotal evidence and just a single example. This isn't a research paper, it's some thoughts I wrote down in a blog post. But really, if you'd have asked me in a user research survey about how things affect me, and my financial situation, I would've answered very differently to if you'd asked me how I felt if prices were being increased for others.

Ironically, the backlash is also user data, and Microsoft reacted swiftly to that new data, reversing course. But seriously, how did they not see this coming? For a few hours, the curtain was peeled back, reminding us the "socially progressive bright cheerful fun Xbox brand" can still trip into a corporatist void if customers aren't paying attention.

Human intuition vs. trending data

I genuinely believe that the people at Xbox aren't trying to make decisions that screw people over, having met many of the people that work there. As an entity, and certainly among the other tech companies, Microsoft is the one that usually seems well-intentioned. But clearly, somewhere along the line, some form of user research and data trends were used to push through these changes. If the data had included figures on the YouTube videos, blog posts, and social media comments expressing anger for the change — despite the fact most people weren't going to be affected by it — they probably wouldn't have done it.

Hindsight is 20/20 of course, and Microsoft, like many corporations, demands perpetual growth. Many tech companies saw a boom as the work-from-home culture bit down last year, and despite that, shareholders will still want to see growth on top of that boom, even when it subsides. It falls on Xbox leadership to find where to extract growth from for Microsoft's insatiable shareholders, and falls on us to call them out on any bullshit as and when the need arises.

This whole Xbox Live debacle just reminds me of a long list of scenarios where Microsoft just ... did stuff, without thinking about how it would affect users, or the perception of the company as a whole. The way Windows Phone was just left to die without any explanation, despite having tens of millions of users. The OneDrive backlash from 2015. The slow, painful demise of the Skype brand. Microsoft often just doesn't seem to get humans.

And sure, I'm not naive. The only data trend that matters really is revenue, and in that, Microsoft is clearly doing something — many things — correctly. Microsoft doesn't have the same loyalty capital that Apple and some other companies enjoy, where their customers will simply shut up and endure stupid decisions, as evidenced by how rapidly people reacted to this whole thing.

I'm sure there were people at Xbox telling Microsoft's finance department that this was a bad idea and that people wouldn't stand for it. It's also easy to imagine that there was someone at Microsoft armed with "data" to "prove" otherwise. How can you argue against data?

I'm certainly no data scientist, and I would hope that Microsoft does account for some of those human elements and data bias. I certainly don't expect Microsoft to take policy inspiration from a random blogger either. I also know that all of this is easy for me to say without responsibility. But, after following them for so long, and seeing the same patterns emerge time and time again, I feel it in my heart that Microsoft should at least consider factoring in some good old human intuition a bit more. Telemetry, trends, and user surveying never paint a complete picture.