Vague Patch Notes: Don’t be the MMO beta defense squad

Plue Brotocol.

Every week, I have to think about betas. I have to think a lot about betas, in fact. More than I ever wanted to think about betas. Far more than I ever wanted to think about games claiming to be in beta when they clearly aren’t or games that have been released but are still hiding inside of early access like somehow it only counts if you say it does… but that’s getting into an entirely different discussion than this week’s topic.

So in this edition of Vague Patch Notes, I’m here to advise you – and by “you” I am referring to members of the MMO community as an aggregate – that it’s time to stop buying into the beta defense. The game that you are playing in beta is, most likely, within sneezing distance of the game that will be launching in its full form. And not only is it not your job to defend the game from people throwing shade at the title in testing. It’s probably actively harmful.

We’ve talked about this before on Massively-that-was as the “beta defense.” The idea put forth by this particular syllogistic argument is essentially that games can change during testing, people not playing the beta aren’t familiar with the game, and things being changed shouldn’t be targeted for criticism; therefore, people criticizing the game just don’t know what’s going on because they’re not in the beta like you.

This is a bad argument for several reasons. The first and perhaps most obvious one is the same reason why fans waiting outside of a theater for a midnight opening of a film may not be the right audience to evaluate a film’s quality. Being in the beta probably means that you’re already interested in the game and hopeful about it being good, which also probably means that you are evaluating it based on how well it adheres to your ideals about what the game was supposed to be.

That doesn’t mean you’re incapable of evaluating it, of course; it just means that you should be aware about your biases going in. We all have biases, and the real trick is accounting for them and letting them inform without dictating your opinion. That’s part of being a journalist.

I just moved here, and I don't want to live here any more.

But the beta defense has problems far beyond your ability (or lack thereof) to properly evaluate the game’s qualities. For starters, and perhaps most importantly, when the game is in beta that title is fundamentally finished.

You’re about to argue that, I’m sure, and you’d be right to point out that beta can launch low on content and with many systems yet to be refined. But in any kind of public beta, it’s important to note that the important part of that phrase isn’t the word beta – it’s the word public. It means that whatever is or isn’t in the game at the time of launch, the developers sat down and decided that yes, this is worth showing off to people who have not yet played the game and/or been iterating upon these systems for months and years at a time.

In classic software parlance, beta means that the product is feature-complete but not necessarily refined or finished. New World, for example, has had a number of test events that have added new content, new systems, and so forth. But if the game moves into its beta, it’s not going to subsequently announce a heretofore undiscussed system wherein you can craft your own magical familiar and summon it as a pet. That system would be in there before beta, even if the initial beta didn’t have everything in place that it would eventually include.

To put it more succinctly, the beta is your visual and public proof of concept. It’s stating what players and fans can expect from the finished game. It’s much more sensible to treat the beta as being more or less what the final game will look like, barring major behind-the-scenes changes that you can’t anticipate.

But that’s the other big problem with the beta defense. It’s assuming that a point of criticism will be addressed in the final game in lieu of any actual announcement of same, and down that path lies the madness of assuming something is going to be fixed just because it’s an obvious problem.

Now… this seems logical, at least coming from a place of good faith. After all, if everyone sees something as a problem and the developers are still in a place wherein changes are not simply permitted but expected, why wouldn’t these elements be changed before launch? The answer is predictably simple, though – it’s very easy for systems that players hate to be the darlings of certain developers who will go to bat, insisting that players are doing things wrong rather than changing the core design.


When you take all of this together, you find yourself starting to acknowledge a fundamental truism. Whatever the state of the game internally, and whatever the intended end point of the game will be, the game in beta – and the information about same – is specifically coming from the development team. Everything being put out in advance comes from the source, excepting the occasional bit datamined out (and even that requires first allowing the public access to the beta information just as certainly).

That means that if people are drawing the wrong conclusion, the problem doesn’t come from the people drawing the conclusions but from the sources of information in the first place. Therein lies the final problem of the beta defense – it’s being the defense for a game that shouldn’t need a defense force in the first place.

If you’re going to be doing the work of defending a game, you should be getting paid to do so. (And as discussed previously in this column, you still shouldn’t be doing anything of the sort even if you are getting paid to be a community manager.) Beyond that? It’s the problem of the game’s developers and staff to “defend” the game, which means controlling the flow of information, possibly being better about communicating, and so forth.

Obviously, the impulse to do this is pretty clear. Just like that audience lining up to see a movie at midnight, if you’re in the beta, you’re probably already invested in this game. That means you probably are inclined to go to bat for it, correct assumptions that you know are incorrect, and otherwise speak up in its defense. It’s an understandable impulse for a game you’re predisposed to like and possibly already do like.

But past correcting things that are factually wrong (and possibly not even then), being in the beta does not obligate you to be a defense squad. Being an observer of the beta certainly doesn’t. Someone being wrong about the game doesn’t affect you in the slightest, and it certainly isn’t something you need to debunk. Even if you know for a fact that Item X is being changed and this person just isn’t aware, you don’t need to devote your time to communicating that fact for people who don’t know it.

Don’t be the MMO beta defense squad. The beta is a test. Your job within the beta is to test things, see what is and is not working, and hopefully figure out if the game is for you or not.

Sometimes you know exactly what’s going on with the MMO genre, and sometimes all you have are Vague Patch Notes informing you that something, somewhere, has probably been changed. Senior Reporter Eliot Lefebvre enjoys analyzing these sorts of notes and also vague elements of the genre as a whole. The potency of this analysis may be adjusted under certain circumstances.

No posts to display

newest oldest most liked
Subscribe to:

Maybe it’s just me, but this seems so obvious it’s like a post insisting that water is wet.

The problem isn’t that the fanbois use the beta defense…it’s that we bother to listen to them at all.

There are people who hate everything about a product irrationally, we call them haters. There are people who love a product irrationally, we call them fanboys.

Once we’ve identified them by their history, why do we spend a single moment engaging with either?

There are also pedophiles and Nazis in the world; I’m not going to waste a moment of my time bothering to try to have a rational discussion with either of them. While love/hate for a video game obviously doesn’t rise to those levels of seriousness, I think the root zealotry is the same flavor.

Not everyone’s opinion is worth listening to, including mine, if I were to display a ceaseless, unwavering, irrational bias in any direction.

Of course, my premise would result in a lot less ad revenue for message boards.

Andrew Ross
Andrew Ross

I get where you’re coming from, but you also have to remember that we get a lot of comments that use this defense, both from site goers and PR/Marketing/Dev teams. These sorts of pieces help make sure that the “truism” is still relevant and can be used to remind people in the industry that, just because they’re doing something “normal” doesn’t mean it’s right or that it sits well with everyone.

Bruno Brito

Problem is when this kind of behavior legitimates zealotry PR from companies.

Thomas Koren

A bigger issue is that many forums(wow for example) have near zero enforcement to reduce the whole “learn to play” “it’s fine” etc crowd, causing this enviroment to become so hostile to criticisms.


A good point, but (maybe) interestingly I’d see it as subtly but nearly completely opposite to the one I was making.

My point isn’t “why aren’t the overarching and ideally beneficent forum gods protecting us from these view points?” its instead why are WE as forumites responding to these view point and interacting with them?

I put it on us as an audience. We are the ones that sustain these commentaries.

Every time someone responds to one of these habitual characters who are well-established fanbois we give them currency. If we engage and (pointlessly) attempt to reason with them, we give them ANOTHER chance to legitimately reply with their generally baseless, propagandistic viewpoints.

We give them a voice.
It’s up to us to stop listening to them.



With that out of the way, i’m surprised you didn’t point out the smoking gun: Can you or anyone else point to *any* beta, closed or open, in the past 25 years which *didn’t* turn out to be the exact launch version of the game, bar extremely minor bugfixes? Because i sure can’t after ~40 “beta’s” and counting. And that’s just the games i was interested in over that period.

The “but it’s beta and they will fix it before launch” excuse needs to die a horrible, violent death. I just don’t understand why people so often so vehemently defend games with that excuse which are at most 3 months away from launch. Hell, i remember quite a lot of people saying the same things about Anthem in 2019, as a poignant example, whose ‘closed private beta’ was like, a month before launch.

Adam Russell

When they have done their last character wipe then you can say its in release.


The biggest offender I see when it comes to alpha/beta/early access titles is performance. So many of these games selling access run terribly, even on the most expensive hardware money can buy, and the defenders insist that it will get better. It rarely does. Because unlike what they’ll say, optimization is not something you do at the end of development. There are no magical tweaks a company can make in the last few months of development that will get a game that’s been running at 30fps for the past 3-4 years to suddenly start running at 60+ fps. More often than not they just benefit from multiple hardware generations releasing between when they started selling the game and when they decide to call the game launched.


My only caveat to your point is that very often a game actually still in beta is running both development code and diagnostics under the surface.
Code optimization is really a thing, PARTICULARLY in graphics performance, which is usually the culprit when people are unhappy with a game being “laggy”…it’s pretty much never going to be the auction house code or the code randomizing npc conversations. It’s almost always about triangles, generating pipes, lod, view distance, etc which, when they’re finally feature-complete and can boil that code down to the tightest possible, can significantly improve for the actual release version.


Those are all good points, unfortunately it’s kind of pointless to say things like that. Many people become overly obsessed with the product in beta/alpha/First Access and the company which develops it to the point of rejecting every negative fact or all negative opinions about that game/company (regardless of how rational the opinions are, even if they are not the facts) and feeling compelled to constantly defend this product and the company everywhere they go, with the “but it’s still in beta and final product will be perfect” being most used excuse by those people. Pointing out how irrational and potentially harmful it is to keep defending flaws in an incomplete product to such people is pretty much useless.


When I was studying computer science, we were taught that being in beta testing simply meant that you had people who were not working for the company, or close friends and family, involved in the test. Alpha testing was when you only had staff or close friends/family involved.

You didn’t need to be feature complete to be in beta testing, you simply needed strangers involved in the testing. This is especially true with more modern agile software development techniques where you want those first modules tested as soon as possible, even if you have months or years left of product development.

Now, as soon as you start charging money, that is when i think you have to stop calling it beta-testing: you are selling a product and that product should be judged on what it is at the point of sale, not on any future promises.

All that said, I keep it simple.

I dont pay for betas. I dont pay for early access. If the game is supposed to be a live-service game like an MMORPG, I’ll only play it when it fully releases and only if there is a clean slate i.e. wipe before the launch.


It’s capitalism: if the customers stopped paying for beta access, devs would stop offering it.

But we live in a very impatient world.


It’s always struck me how similar it is to a dysfunctional relationship. This time he won’t lie to me. This time will be different. He’ll change, I know he will. I wish everyone would stop interfering, they don’t understand what he’s really like.

I can’t turn around after all this time and admit I’m wrong, he’ll come through for me this time. This time, for sure.

Happy new year, Star Citizen wives’ club!


I think it’s important to remember that constructive criticism helps a game.

A good development team will listen to feedback (even if ultimately they decide not to implement it) and continuously strive to make their game better.

Criticisms that come up in beta aren’t likely to disappear at launch. Fans who feel they need to defend their game of choice against any negative feedback and treat every criticism as just an attempt to harsh their mellow may inadvertently harm the game by insulating developers from hearing hard truths.

Constructive criticism is an opportunity to improve.

Kickstarter Donor
Peregrine Falcon

Yes, constructive criticism can theoretically help a game to improve. In actual practice? Probably not.

First of all, attacking other people on the forums, the reddits, or the comments section isn’t ‘constructive criticism’. I’ve been in a couple of betas over the years and I’ve seen far more of that than of all the other types of posts combined.

Secondly, game developers don’t pay attention to actual feedback anymore. Sure, if a thousand customers flood their forums with flames about a particular issue, or start demanding refunds, then they’ll probably listen. But, if you start a thread with a bullet-pointed list of issues/bugs you’ve found, and suggestions on how to fix them or improve the math in some way so that it’ll do a better job of what the developers want it to do then… the developers will probably never see your thread, and they will certainly never respond to it.

This is why I don’t have anything to do with betas anymore, period. I’m not getting paid to test their product, and they won’t listen to anything I have to say about anything. Basically it’s just a waste of my time and yours.

Danny Smith

Every time the modern “i bought early access and can’t make bad choices, thats unpossible!” hyper aggro beta defender crowd springs up when something looks rocky i always think of this.

From the halcyon days of the Tortanic and its “losing half the players means theres room for more on the servers so we will hit 17 million players in the first year!” to Fallout 76’s “the bugs are all lies made up by hater trolls, Bethesda does NOT make buggy games” and all the screencapped cringe inbetween some things just never change and people never learn because this time?

This time they really are going to fix it. They said so. Company’s can’t lie to me. Thats illegal.

John Mclain

Completely agree. I hate how companies and players alike use the term “beta” as a shield for broken buggy games. As soon as a game is being sold to you, or has a working cash shop in it, it is no longer a “beta” it is a live game.

Danny Smith

I’m still surprised how many people fall for it. I remember back at the launch of the PS3 when sony had a rocky second life knock off called PSHome on it Penny Arcade said “Why does it still have a beta label when its been live for a year with microtransations? because you aren’t allowed to criticise it, ‘lay off its just a fuckin’ beta man!’ is the publishers ultimate gotcha”.


As soon as a game is being sold to you, or has a working cash shop in it, it is no longer a “beta” it is a live game.

Oh yea, I absolutely agree. The moment you can buy the access to the game and log into the game right after buying the access to it – all the excuses about “but it’s beta/alpha/First Access so it’s ok for the game or the trailer used to promote this game to be low quality and look like shit” become invalid, especially considering every company is always free to call the game’s development stage any way they want to and they can keep calling it a “non-final product” for as long as they want to while keep charging people money for access to it.