I’m wrapping up my on-site E3 coverage this year with a meeting more about the industry meta than about any one game. The interviewee? Jake Parmley of Red Fox Insights. We’ve covered the firm before; its a video game market research company that claims to tap into about 70 million players world wide thanks to surveys built into partner sites reaching general gamers and niche gamers alike. That means surveys about FPS games will be found on, say, a review page for PlanetSide 2 rather than a guide for World of Warcraft.
My meeting was something that most general games may ignore, but those of us deeply invested in it are at least curious about it. Be warned:Â This is going to be one of those “sausage factory” type articles that will expose you to the inside of the industry!
Why even go to E3?
As Parmley has argued that E3’s role in the industry has been under fire for a long time, but it’s clearly still serving a purpose. In fact, Red Fox was at E3 partially to talk to some of the smaller gaming studios. Any event will bring gamers out to gawk, but smart companies know that different conventions have different strong points.
One company Parmely met with, for example, had used E3 for finding a publisher the year before and PAX to expose itself to the public. At PAX, exhibitors focus more on demos than reveals. At E3, though, companies are confident enough to simply announce a game and show a hands-off demo, seeking exposure rather than feedback. In fact, while PAX seems to be about breaking down barriers to get people to play games, Parmley says that E3 is much more about restricting them. There are a lot of invitation only demos, pre-recorded demos to view, and simple trailers at E3.
Big companies that can host their own events have different goals too. Parmley cautioned against reading too much into a studio’s attendance. “Attendance, or lack of, is more a result of their own needs, rather than an inherent flaw with an event or the audience of that event,” he says. “This explains why teams drop out one year and return later.” A studio running its own event gets to choose who comes in the door, and that often means not critics but influencers who will help promote a game and focus more on hype than substance. Emotional response sells better and is often more entertaining than a well-formed thought, and that has a value that’s easy to read in dollars. By catering more to influencers, E3 could potentially seem more relevant as Parmley suggests, but it also risks becoming little more than an advertising spectacle. Personalities who aren’t bound by ethics are easy to impress and buy.
How to demo at E3
Parmely suggests that if a developer’s goal isn’t just to make an announcement but to try to get everyone to play its game, it should definitely consider keeping its demo length under ten minutes. This was interesting to me as Nintendo’s 2015 floor staff last year noted the company’s goal was to increase demo times since attendees wanted to see more. Last year’s Star Fox Zero demo was about 12-15 minutes long, while this year’s Zelda demos took a total of 35 minutes. (Nintendo might be an outlier, though; I suspect it uses E3 as a kind of industry demo opportunity, as its staff always seems much more interested in feedback than simply processing lines.)
This is why MMORPG tutorials at E3 tend to suck. Many are often newbie experiences, but newbie experience may not be good for an audience that knows the genre. It’s why a demo invitation MassivelyOP gets that recycles PAX content is a supreme let down, not because it’s something everyone gets to try but because I’d wager that the newbie experience is still often the weakest part of the MMO experience in most games. Worse yet is when the demo is unstructured and unguided, as simply jumping into a game with absolutely no knowledge of its strengths weaknesses causes both pros and casual gamers to walk away. While you can use a system that potentially restarts a demo if players wander too far off track, it’s more frustrating than it is helpful.
That leads to another note about demos: Don’t talk flaws! People want to see what you brought, not have someone influence our view as we play, especially in a negative way.
As much as I personally hate it (as media and a fan), Parmley actually suggests that companies use one year at E3 to announce a title without giving a playable demo, then follow-up the next year with the demo. While the gap’s probably too large for your average player, I’d wager Sea of Thieves benefited specifically because of this advice.
Swag at E3 probably needs a rethink too, especially if a company has the money. Parmley and I agreed that simple t-shirts work well for for both events, as it turns even cynics like me into a “bunch of moving, talking, marketing signs.” Rotating shirts on different days to increase return visits is a deviously successful strategy we saw work, but this can be done with cheaper buttons too (though may be more effective with the PAX crowd).
Red Fox’s insight process
Due to my own interest in games and research, I was curious about Red Fox’s research methods and aims, even if they’re specifically for business purposes. Parmley noted that developers and publishers are “excited” about data-based research but are expecting certain results. The company is committed to representing data fairly, so if the data contradict, the developers just have to accept that. And while the data collection method and potential reach is any serious scientist’s dream come true, Red Fox’s survey widgets use ad space, which may mean they’re blocked by ad blocking software. This is probably why I’d never heard or seen anything from the company until we covered its research ourselves as news.
The data gathered from Red Fox’s research questions are fascinating. Surveys are only counted once, even if a user takes the survey several times, so when people do participate, they’re not easily able to skew the results. But survey-based research depends not only on people understanding the questions asked of them but that they give honest answers. Even when they’re honest, the fact of the matter is that people may believe they’re giving an honest answer when they actually don’t recognize the truth. For example, I identify as a PvP player, but if you were to look at my in-game actions, I do a lot of things that are non-combat related or that help me avoid PvP. Ten years ago, I might not have realized that about myself.
This is important because Red Fox is focused on business and marketing, not the rigorous peer-reviewed science I usually cover for MassivelyOP. Admittedly some of the logic used may be faulty enough for even readers to catch, but the numbers backed from hard data collected from large pools of fans from various walks of gaming are hard to ignore. It’s almost the inverse of what we get from peer-reviewed studies, and it might be interesting if the group found a way to partner up researchers.
