Yahoo has an anti-abuse AI to seek and destroy trolls

With a 90 per cent rating, Yahoo’s AI is striving to take hate speech off the internet.

Meryl D'Souza August 2, 2016

In May, Jonathan Weisman, deputy Washington editor for the New York Times, wrote about his experience as a victim of harassment on Twitter. It started after Weisman tweeted a Washington Post article about Donald Trump titled "This Is How Fascism Comes to America"

“Hello ((Weisman))” @CyberTrump tweeted in response. Inquisitive, Weisman asked the twitter handle to explain the parentheses. The account holder replied with: “It’s a dog whistle, fool. Belling the cat for my fellow goyim.”

The (((echo))) symbol is used by white supremacists to single out Jews, to its online database of hate symbols. "The anti-Semitic hate, much of it from self-identified Donald J. Trump supporters, hasn't stopped since," Weisman wrote.

The trolls kept having a go at Weisman until he quit Twiiter two weeks since the episode. The parentheses are key here since the symbol is not searchable on social media and search engines strip punctuation from results. Which essentially means that trolls go unnoticed as they harass and threaten their victims. 

It was important to start with that because contrary to popular belief, trolls aren't a harmless wisecrack-spewing bunch. Some of them come with a much bigger agenda. Some of them are dark anarchists who revel in the distress they cause their victims. 

Yahoo’s anti-abuse AI

This is what makes Yahoo’s abuse-detecting algorithm so vital to our time. Yahoo’s algorithm seeks out and can identify hate speech in comments in 90 per cent of test cases. 

The company used a combination of machine learning and crowdsourced suggestions to build the algorithm after trawling the depths of the comment sections at Yahoo News and Finance. However, unlike other AIs Yahoo’s algorithm doesn’t look at specific words, instead it looks at a combination, as well as overall post length, punctuation and other metrics to determine what constitutes abuse. 

While the algorithm hasn’t been used outside of Yahoo’s dataset, the company is working on maintaining the algorithm’s efficacy and perfecting its detection techniques. This is just the start, but at least someone is taking charge.