Twitch is stepping up its anti-harassment efforts with a new tool that employs machine learning to detect those who are attempting to circumvent restrictions. It’s the company’s newest tool to counteract hate raids, in which trolls flood broadcasters’ chat rooms with vile remarks.
Suspicious User Detection is a new tool that may identify individuals as “likely” or “possible” persons who have gotten around bans on a streamer’s channel. The tool’s machine learning algorithm identifies prospective evaders by examining factors like their activity and account features and comparing that data to accounts that have been banned from a streamer’s channel.
Messages from “likely” evaders will not be sent to chat, but they will be visible to broadcasters and their mods. Streamers and moderators can opt to either monitor or ban a suspected ban evader, which adds that person to a monitoring list and displays a notice next to their name indicating the monitoring (as demonstrated in the GIF below). Messages from “possible” evaders will display in chat, although broadcasters / mods can choose to have such messages blocked as well.
Suspicious User Detection will be turned on by default, according to Twitch, although streamers may alter or turn it off if they choose. Streamers and moderators can also choose to manually monitor users they suspect.
In a statement to reporters, Alison Huffman, Twitch’s director of product for community health, stated, “This tool was motivated in large part by community input around the need for improved solutions to limit ban evaders.”
“When we spoke with moderators about their frustrations, we realized that it can be difficult to tell if a user who talked anything that broke their channel’s standards was a destructive, repeat harasser or simply a newbie viewer who hadn’t yet acquired the channel’s customs.” As a result, we created this tool to provide additional information to moderators and creators about suspected ban evaders, allowing them to make more efficient and informed judgments in their channels.”
Suspicious User Detection appears to have the potential to help silence unpleasant people, especially when combined with recently announced restrictions that allow a streamer to require phone or email verification for chat accounts. However, it remains to be seen how efficient Suspicious User Detection will be in practice, as well as if ban evaders would be able to work around the technology.