An investigation has exposed hundreds of thousands of online hate profiles, leading to a call for tougher regulation of online platforms.
A “foul trove of racial hatred” was uncovered on Twitter and Facebook as well as amongst the gaming community through research conducted by the UK-based Centre for Analysis of the Radical Right (CARR).
It comes six months after a public outcry at the abuse of England footballers Marcus Rashford, Jadon Sancho and Bukayo Saka, who were targeted online after missing penalties in the Euro 2020 final at Wembley.
What was found on Twitter?
During their two-day analysis carried out in January, CARR researchers looked for profiles using simple words and phrases as indicators of “systemic failure”.
On Twitter, they found around 300 users or profile names derived from a racist phrase, including the N-word, dating as far back as 2009.
Dr Edward Gillbard, who carried out the research, said the majority of accounts had fewer than two followers and were following less than two accounts.
How a student’s black foetus illustration seeks to diversify the medical world
Wall Street regulators investigating Trump $1bn social media deal
Lush quits Facebook, Instagram, TikTok and Snapchat amid concerns over online harm
He added that it would “appear there is no automatic moderation being performed by Twitter” in terms of analysing accounts for offensive usernames.
Twitter said the accounts had been “permanently suspended” for “violating our hateful conduct policy”.
A spokesman said: “We acknowledge and want to reiterate our commitment to ensuring that Twitter doesn’t become a forum that facilitates abuse and we continue to examine our own policy approaches and ways we can enforce our rules at speed and scale.”
What was discovered on Facebook?
On Facebook, dozens of offensive profiles, including 83 variants of “hate (N-word)” and 91 on the Holocaust were identified.
Other profiles included the name Adolf Hitler and other high profile Nazis, as well as the names of mass killers such as the Christchurch mosque attacker in New Zealand.
By changing the spelling or inserting spaces and special characters, profiles seemed to be able to fool moderation systems, Dr Bethan Johnson, who found the accounts, said.
“It may be that when users set up profiles with names that clearly mock and flout community standards – from ‘Jewkilla’ to ‘Nate Higgers’ – they are telling Facebook what kind of user they will be, what kind of ideas they bring to the platform, and the reality is that is far from community-orientated,” she said.
A spokesman for Meta, the parent company of Facebook, Instagram and WhatsApp, said hate speech was not allowed on its platforms and the “violating” accounts were removed after being flagged.
He added: “If we find content that violates our policies, including the use of symbols, emojis or misspellings attempting to beat our systems, we will remove it.”
What about gaming platforms?
An analysis of the digital gaming service Steam revealed more than 300,000 offensive profile names.
Of those, 241,729 were anti-black, 44,368 white supremacist, more than 28,000 neo-Nazi, 8,021 anti-Semitic, 5,607 homophobic, and 168 anti-Muslim.
On the game Fortnite, more than 100 racist and far-right extremist profile names were found and 34 were identified on Rainbow Six Siege, 18 of which were active.
A spokeswoman for Fortnite developer Epic Games said many of the offensive usernames are no longer in their systems and action has been taken against additional accounts provided.
“Usernames that include vulgarity, hate speech, offensive or derogatory language of any kind are in violation of our community rules,” she added.
It is understood that the Rainbow Six Siege profiles have been reset with randomised names and any offending pictures have been removed.
A spokesman for the game’s creator Ubisoft said the company “does not tolerate any form of bullying or harassment”.
The firm takes “concrete actions” to tackle “toxic” behaviour, and violations of its code of conduct could lead to sanctions, including bans, he said.
While automated processes were not “foolproof”, teams are constantly working on improving them, he added.
Follow the Daily podcast on Apple Podcasts, Google Podcasts, Spotify, Spreaker
What has the reaction been?
“Finding a foul trove of racial hatred on social media is still shockingly easy,” said director of CARR, Professor Matthew Feldman.
“It makes you wonder what the point of moderation is when some of these obvious, overt and in some cases violence-inciting accounts can go literally years with no consequences, and certainly no moderation.
“This material is disgusting and makes it seem that platforms just don’t care enough to address this running sore.”
He added that platforms had a “duty of care” to users but only government regulations and the threat of tens of millions in fines would bring change.
“Otherwise, these platforms will stay reactive – badly – rather than proactive in taking down hateful extremism,” Prof Feldman added.
Danny Stone MBE, chief executive of the Antisemitism Policy Trust which jointly published the report, said: “Six months from the Euro finals, a year from the insurrection at the US capitol, but the story remains the same – social media companies profiting from the sale of our data but failing to properly protect people from harm.”
He added: “I hope the forthcoming Online Safety Bill, and legislation across the world, will force social media companies to better look after their users because they appear to be in no hurry to help.”
