ToxMod is a new moderation tool that recognizes and reports voice chat hate speech in real-time

    
33

Moderating a game’s text chat isn’t easy by any means, but at least there are a variety of tools and algorithms that try to make managing it a bit easier. That’s not quite the case for voice chat, which can not only be more immediately impactful but also be harder to act upon quickly. Modulate Inc, a technology company using artificial intelligence to improve online voice chat, hopes to change that with the release of a tool known as ToxMod.

ToxMod is described as “the world’s first voice-native moderation service,” letting game developers detect toxic, disruptive, or otherwise problematic speech in real-time, and respond swiftly and as they deem appropriate. ToxMod utilizes machine learning models to understand not just what each player is saying, but how they are saying it, taking into account emotion, volume, prosody, and more. These features help ToxMod to focus less on whether something is being said in a moment of frustration and more on the start of something more disruptive. ToxMod even learns more about a game’s community the more it’s utilized on top of its core voice recognition algorithms.

For those who are concerned about Big Brother levels of listening in, the software creators insist that ToxMod processes all data on each player’s device in real-time, preserving player privacy and only sending audio off-device to mods when toxicity has been detected.

source: press release
Advertisement

No posts to display

newest oldest most liked
Subscribe to:
Turing fail
Reader
Turing fail

I often forward voice-to-text translations to the original senders so we can share a laugh. The tech we use isn’t anywhere near perfect, despite my employer’s rep as a tech-savvy company. These translations are sometimes NSFW… of course I’m sure ToxMod is always 100% spot-on.

Reader
Schmidt.Capela

The red flag for me is doing this on the player’s device due to privacy concerns. Anything that is fine for the AI to eavesdrop would also be fine to send to a central server to be monitored; anything you should refrain from intercepting and recording, you should also refrain from using an AI to mine for trigger words.

As for what I think should be monitored: public channels, group channels when at least one player was added through an automated LFG-style tool, and unsolicited private communication (as in, one player started a chat with someone not in their friend list). On the other hand, channels where everyone is on each other’s friend list should never be monitored regardless of their nature (private channels, group channels, etc).

BTW, I like one idea that is apparently being used on the PS5: automatically recording the last few minutes of voice-chat and offering players a fast and easy way to send that to moderators if there was any abuse. This makes it far easier to punish players with abusive behavior.

Reader
Danny Smith

This is a terrible idea. Language is not static, especially online. There are plenty of internet slang, insults and the like that only exist BECAUSE they started as a way to get around some form of filter. Just look at the famous case of 4chan trying to ban the term “cuck”. Or Twitch’s recent actions that caused usage of terms they wanted to ban to skyrocket.

Nothing makes people want to curse and shit talk like being told they aren’t allowed. Its like how every year language courses at universities need to go “okay and heres the new words that mean something absolutely foul that didn’t exist last year, take out your books and start at… ‘big plunbus’ please” because you can’t control this no matter how hard you try. Language as a construct exists to give meaning to something, you try and wrongthink police it the words change but the meaning stays the same.

and thats not even taking the regional stuff into account. In most of the world ‘spook’ means ghost or spy, in america its apparently a racial slur for black people. Does that need policing? its just something you can’t automate and forget about.

Reader
Dankey Kang

Discord conv: “Yeah man, got a big bottle of Lucozade here”

Error; Lucozade is on the forbidden word list. Your account has been closed, bank account shut and the police have been contacted. Your internet will be disconnected in the next 20 seconds. Have a nice day

Reader
MaynardKeenanX

Creepy future….no thanks…

Reader
Soy

Maybe we could nail text chat moderation before moving to this. A day in WoW gen chat is evidence we’re not there yet.

Reader
Zuldar

The idea is fine, text chat is moderated no reason voice chat can’t be as well. You can always setup your own Discord server if you really want to avoid censorship.

I’m more concerned about the implementation and if this ends up being used beyond just gaming or for political purposes within a game.

Celestia
Reader
Loyal Patron
Kickstarter Donor
Patreon Donor
Celestia

Can’t wait to discover all the ways this can be exploited and bypassed before it is inevitably abandoned.

Reader
Tuor of Gondolin

Real time, online chat monitoring and censorship. Big Brother would be so proud.

Reader
Bryan Correll

Do you want Ultron? Cause this is how you get Ultron.