Massively OP’s guide to understanding video game research

April 27, 2016 12:00 PM

I noticed two strange things in the comments section of our lengthy Exploring the Video Game Debate series: People were angry at research that supported what they were trying to argue but hadn’t read well enough to see it, and people discredited the research because their personal social circles had vastly differently experiences (but also, often, didn’t read the research).

And you know what? That’s normal. While game research isn’t nearly as important as biology, a recent BBC article reminded me the internet has made it quite easy for people to plug into communities that reflect and validate their common opinion, sharing and replicating (mis)information again and again. While most of the core MOP community is between 25-44 and revolves around a shared fandom, it’s still possible to be affected by poor-quality input, as seen with Microsoft’s recent teen AI, Tay, and her venture into Twitter.

However, using some semi-recent research about a 14-year-old learning English through World of Warcraft, we can see how to approach research without being affected by bad data (or needing a research degree!).

Use your head, not your social identity

A three-part study by German researchers recently — and unsurprisingly — found that people judge scientific research based on their past experiences and social identity before trying to understand the actual studies. Though the research was about gamers and views of violent games, the researchers said that the same bias is true of most people and (sub)cultures.

While forming opinions based on past experience is essentially the basis of scientific research, basing beliefs on your social/cultural identity is quite dangerous. Remember, video game addiction still isn’t considered a “real” problem because psychologists argue it’s so tightly bound to culture. However, psychology is a “soft science.” There’s a difference between hard sciences and soft sciences; most people probably can’t explain the exact biology behind how their heart works, but they accept that doctors know enough to operate on it. However, if a psychologist says that games can make you desensitized to violence (which is true to a point), lay people who play video games and may not think they are violent will react (sometimes violently). Because soft sciences often use data gathered from personal experiences, people often feel their own experience or knowledge can disprove it.

The problem there is that science is based on past experience and generalizable results, meaning the findings have to apply in multiple and somewhat similar contexts. For example, I spent most of my teenage life never meeting anyone who played the Madden game series. When comparing my experience at E3 with videos of the event for the public, I saw that live Madden reveals got nearly no applause, while event recordings included canned cheers. However, the series clearly sells — but to sports fans, who aren’t well-represented in my personal social circles.

A simple way to check my bias was to google Madden and Sports Illustrated, a sports magazine popular enough that even this die-hard geek knows about it. Sure enough, there are tons of hits referencing the two, showing that, yes, there are some important institutes supporting the Madden series, even if they are barely on my radar. Checking with mainstream communities you don’t often associate with is a good way to mentally counter your bias and avoid overreacting.

In the research I mentioned earlier, the researchers suspected some gamer-participants may have rushed to finish an article they condemned for discussing games and violence, even though the findings did not support a link between the two. The researchers did another study that specifically included an attention check test to correct for that problem and weeded out those who failed it. Again, those who identified the most as gamers were biased against the research from the start and often at the end, some wholly distrusting science simply because the article potentially challenged their identity.

The problem is that people are pre-wired to rebel against information that may threaten their identity, and social media doesn’t help. However, if you know about that pitfall in advance, you can simply remind yourself of your bias and then actually consider what the research says.

Understand the research process

Now, don’t mistake me: Please don’t believe everything you read! It’s good to be critical. Just keep in mind that real research is “peer reviewed” — it has to make it through an obstacle course of other experts in the field — sometimes rivals — before it can be published in the more respectable and highly rated journals. Think of it like a peer review at work: You are both a potential ally and threat, so making it through that process isn’t always easy. Yes, Dan in accounting is a jerk that holds a grudge against you for spilling coffee on his favorite tie, but unless your work place is truly awful or corrupt, there are enough opinions within the company to explain what you do and how you work with others. Research, in theory, gets the same treatment.

Because of this, research also needs allies. By mentioning a long list of other studies (we’re talking pages of research titles!) that discuss the same topic as the conducted study, the researchers are essentially building a sturdy foundation to show that their study is based on other grounded studies that also went through the same peer-review process.

In addition to that, research also needs to be upfront about potential weaknesses and flaws and mount a defense of them so Dan can’t bring them up to delay publishing the findings for another year and holding up your funding. If you ever suspect something’s critically wrong with a study, looking at the actual article should reveal it.

At this point, most people will want to use personal experience. When this happens, ask yourself this: What will the opposition say? Think of my Madden example. It’s quite simple to reject if anyone says, “My experience is different!” Personal experience is just that: personal. When using personal experience, you want to find the parts of your experience that can be generalized since research does the same.

Use your experience

Let’s take a semi-recent example. Drs. Liss Kerstin Sylvén and Pia Sundqvist previously mentioned a 14-year-old boy who learned English through WoW. Dr. Sundqvist later wrote a short study on the boy. Without reading the article, many people will notice that it’s simply one person and question the results, and in some ways, that’s good. We want large group studies whenever possible. However, gaming still has a bit of a negative view in the world, and research on it is far from mainstream, which means smaller studies are still the norm out of necessity.

Sundqvist used a semi-structured interview, meaning the interviewee was encouraged to talk freely with little prompting, so researchers could later discuss the “stories” to highlight findings in relevant research, drawing conclusions to support or detract from other studies. The interviewee was a 14-year-old Bosnian boy who moved to Norway at age 4 and then Sweden at 6. He began learning English in third grade, and in sixth grade he went to a school that used English as the classroom language for many subjects (Swedish law permits only half of school time conducted in a language other than Swedish), so he was basically already multi-lingual and being primed for it, as even his school required him to learn another language (he’d chosen Spanish). He knows a little Croatian and Russian as they’re related to the Bosnian language. He wants to be a doctor like his parents, got to choose his vocational school, and actually approached the researchers on his own.

The boy had recently taken a test that indicated that he would have been able to pass a college-level English test. He’d specifically noted that he plays games for escapism and games allow him to do things he wouldn’t do in real life. The stories were what grabbed him initially, so he started off with single-player experiences before going online. Reading and writing in-game with others helped him correct some languages mistakes. For example, for some reason, he’d thought “s” was pronounced as “h” so “selling” became “helling,” something that his follow players helped him correct. This shows he has self awareness of learning, something that can give learners a very critical edge. He also thinks it’s “awesome to learn new things,” so he likes school.

So here’s what I need to consider when comparing the subject to my own experiences. First this, isn’t a normal kid. It’s difficult to compare his success story with the potential of a mono-lingual mid-western American trying to learn Swahili. Heck, reading this was personally a bit discouraging for me, as I’m an English teacher in Japan and have really struggled to improve my Japanese speaking and writing skills. Like the boy, I have some multi-lingual history, am aware of my learning habits, started off (in English and Japanese) playing games for their stories, and I even live in a foreign country that speaks my target language.

There are a few differences, though. As much as I love Japan, I generally haven’t imagined making this my permanent home, so my future job may not need me to know Japanese. The people I talk to rarely correct my Japanese (and the ones who do rarely get to see me). In fact, I felt like my gaming partners were more worried about trying to communicate with me than actually communicating, creating a stressful atmosphere that prevented any kind of escapism or ability to focus.

Generalize (or ungeneralize) your findings

This is where things get interesting. When you start reading more research, you can draw connections to past articles you’ve read. While personal experience is a decent starting point, simply being able to mention other studies that discuss your experience can help you better understand why your experience may or may not match what you’ve read.

For example, the same journal issue with Sundqvist’s study had another one on Japanese learners using radio programs to study English. They had a lot in common, but this study discussed motivation. Of particular interest was the breaking down of motivation types. The Bosnian boy had several types of motivation that seemed to lead to positive language acquisition: desire for international information exchange, willingness to communicate, confidence in communicating, and motivation to study alone. I, on the other hand, have low confidence in my Japanese ability, a key issue that saw other participants perform more poorly than those who were confident but had lower language skills.

Drs. Kowert and Bowman also wrote something around this time about the role of games in learning through creativity. They mentioned that creativity has been shown to help develop several psychological benefits, such as with overall mental health and problem solving, so gamers often have higher perceptional and cognitive abilities (which is supported by Naval Research). I might have higher perceptional and cognitive abilities as a gamer, but unfortunately my personal life in Japan negatively affected my mental health (mostly confidence), which is another large strike against the learning benefits I should have.

The most obvious way to see this is with a state of mind games can cause: “flow.” I’ve thought of it as “being in the zone” or “one with the game,” though it’s not limited to games. During “flow,” you ignore unimportant, non-related factors, home in on your required tasks, and suppress your inhibitions and self consciousness in a bid to perfectly respond to feedback. It’s not that there’s no spoon; there’s an idea of a spoon, and you’re creating, bending, and destroying spoons like some sort of god. It’s great when you can do this in a game since the perfectly balanced game primes you for this state, but it’s quite a feat to pull this off in real-world tasks. It’s even harder when you’re focusing more on personal problems and consciously trying relax among other people who are stressed out, which I experience with both gamers and non-gamers.

That brings us to identity construction. Games should help the player form an identity with their avatar. This is what helps the player distance him or herself from reality and consider problems from the perspective of the game world. My problem was that even in online games, my character’s issue was with communication and being an outsider. I had no gamer friends I brought into my online adventures with Japanese players, and there was more of a focus on communication than playing, to the point that I felt speaking to people in a PvP game and revealing my foreignness actually made me a larger ganking target.

As much as I wanted to be able to write-off the Bosnian boy’s success, I could see how it was useful: He had a good mix of skills that theoretically would help him learn a language, and it seemed to work. Although I possess some similar qualities, I was missing ones that would have made utilizing those qualities beneficial.

Read between the lines

Again, you don’t have to trust everything you read. It’s important to be critical, but remember that peer-reviewed research goes through a hell of a process. It’s tempting to use personal experience to try to debunk science, especially when the headlines threaten your social identity, but unless you can apply it outside your immediate social circle, it may mean your experience has limited generalizable value — just as mine did. Always ask yourself how your argument can be applied more broadly. For example, I’m often critical of modern game communication research that focuses only on text. My friend’s dad doesn’t really “get” her gaming habits, but the guy is serious about Skype, so how can modern research talk about communication in games without discussing voice chat?

By being able to check my bias, trust peer-reviewed research to be vetted against obvious fraud, and compare my thoughts/experiences with other research, I can make use of professional academic research to help me understand the world around me, digital and otherwise, without regurgitating some of the sludge that clogs up intelligent conversation on the internet.

Enjoying our gaming science articles? Let us know you want more and get caught up on Andrew’s entire series on The Video Game Debate.