A new Japanese study in the acclaimed science journal Nature suggests that Pokemon Go players experienced a drop in “psychological distress” because of the game. The paper gives psychological distress a specific definition, but it’s easier to explain it as the amount of “vigor” someone has compared to depression, anxiety, and fatigue.
While the study had a control group of non-players and used a total sample size of 2,500 fully employed Japanese workers, at this point in the research game, I’ve started to become less impressed. While I’d love to sing praises of POGO, there’s a reason most people look at the game as a fad, and the research here only reinforced this. Let’s take a look at the study and what it really means.
Research history (problem) repeating itself
I’ve tackled this subject this year once already: POGO research often depends on people who played the game about 30 days, which also happens to be the cutoff for when people start to quit the game. To make matters worse, this research team behind the original study (once again using surveys collected online) also conducted the POGO-related follow-up in December 2016, about five months after release. Anyone who played a month was lumped together, so people temporarily riding the hype train were included with long-term players, without any published figures separating the newbies from the vets. This is key because as even the researchers noted,
Previous studies13, 14 have indicated that an increase in physical activity was no longer observable after several months.
As in the previously mentioned Microsoft Band research, the actual Pokemon Go player pool was significantly smaller than the total participant pool, representing only about 10% of the 2,500 participants. They were also younger than the control group (again), and women were (again) in the minority, though not as dramatically. Â
After looking at pre-POGO surveys and post release surveys, the researchers found that the psychological distress levels among the POGO players fell the most and were statistically significant when compared to non-players. It should be noted that physical complaints went down, though those weren’t deemed statistically significant.
Further more, both players’ and non-players’ work performance scores also fell, with the players losing a bit more than non-players, but once again, this wasn’t considered statistically significant. I would assume the winter months might account for this perception, as the snow keeps people in a little more and not all buildings have good heating.
However, the POGO players’ health issues were reduced, and if they’d been out in the cold (hello ice-related incidents!), I’d think they would have gone up. The team was looking at “dizziness, arthralgia, headache, neck and shoulder pain, backache, eyestrain, aspiration, malfunction of stomach, loss of appetite, constipation and diarrhea, and insomnia” in this category. Although some of these are related to general health and diet, some can be seasonal and more salient. While some people faint in the heat or humidity, and spring allergies drive people nuts, I remember winter was when co-workers were breaking bones and complaining about aches and pains, not to mention anecdotal stories of increases in sickness which, despite being unscientific, would be things people complain about in a survey. Again, these were statistically insignificant, but exploring the difference might be useful, and that was one of the bigger issues I had with this paper.
Branching out
On the one hand, this is a paper published in Nature, a well-respected scientific journal. On the other, just a quick read through it and the graphs made me feel like this study may have been less about Pokemon Go than about the game’s audience. It’s like saying people who play polo tend to be healthier than those who don’t when these same people also have significant wealth and free time that allows them to do it. In other words, wealth is a confounder.
For example, the researchers didn’t mention anything about how the game is most popular with people living in households making between about (the Japanese equivalent of) $30,000-$80,000 annually. People in management seem to play less than people with clerical jobs. Essentially, it seemed as if most of the players were middle class. To be perfectly blunt, I consider gaming cost-efficient entertainment and good for people who want to stay home for their fun. But POGO asks these people to go outside, and the extra exercise is generally assumed to be connected with better health. Again, if the research showed us how long people had been playing, the results might be more significant feeling, even if they just reinforced popular thought.
The research is the first of its kind in terms of POGO and psychological stress relief, and it started with a great review of the issues the game’s brought to light; I didn’t know there was an increase in mosquito borne diseases associated with the game, for example, but it’s also not surprising. With any luck, we’ll start seeing some follow-up research about the effects of the game past 30 days, or maybe even on some other AR games.
