Stern researchers found that extremists across the political spectrum often use high-profile incidents to create divisiveness on social media platforms and perpetuate violent rhetoric. The report, released last month, found that they take advantage of algorithms and privacy features on platforms such as Telegram and X to optimize reach without heavy moderation.
Researchers worked with Tech Against Terrorism — an independent organization launched by the United Nations to fight online violence — to monitor online activity from March to June, specifically focusing on communities they deemed “far-right, far-left, violent Islamist, and nihilistic violent extremist.” Stern policy adviser Mariana Olaizola Rosenblat and senior research scientist Luke Barnes, who authored the report, focused on how extremist groups could use unique algorithms to better target individual users and lead them toward holding more polarized and often violent views.
“The algorithmic ability to fashion your own ideology makes it a lot easier to, as a former FBI director mentioned, come up with your own salad bar of extremism,” Barnes said. “So you want a little bit of the far right, you want a bit of the far left, you want a bit of something completely esoteric — that’s much, much more possible nowadays than it was even 10 years ago.”
One of their main findings was that extremist groups strategically vary their content across different platforms. They use mainstream apps like Instagram, TikTok or X, then direct users to encrypted platforms such as Signal, Discord or Telegram, where they can share content with minimal moderation.
However, Barnes noted that those apps can also serve as means to rally for “pro-democracy protests” in places where organizers’ identities and safety may be at risk. Because of this, it’s challenging to suggest drafting regulations that don’t infringe upon free speech concerns. He also mentioned that X, one of the more public platforms they monitored, was mainly used to sway others toward a certain political ideology due to its high viewer engagement.
“You can’t solve the problems of digital harms by trying to ban every digital harm,” Barnes said. “But at the same time, that high threshold for threats and terroristic content that the United States has — it results in a lot of hate speech. It results in a lot of doxxing, of malicious activity online.”
Rosenblat said that because TAT’s strict policy prevents analysts from interacting with users, she was unable to access certain encrypted groups and instead gathered data in the aftermath of acts of violence.
Researchers also found that extremist groups are most active — and often most effective — following high-profile acts of violence, especially those with political undertones. Appealing to preexisting narratives, the groups can exploit heightened emotions to scale the impact of violent incidents. The report comes months after the assassination of former Minnesota State Representative Melissa Hortman and her husband, as well as prominent right-wing figure Charlie Kirk, amid rising political violence and polarization in the United States.
The report recommends that social media platforms more precisely define threats, make it easier for users to report violent conduct and responsibly collect data to flag potentially dangerous users. It also suggested that policymakers create a distinction between protected speech and illegal terrorist threats, draft guidelines for the type of information that social media platforms can share and support research that counteracts violent narratives.
“Every platform has to strike a balance, and governments too, in the regulations between privacy, freedom of speech and safety,” Rosenblat told WSN. “But each platform should have a responsibility to moderate or mitigate against harm.”
Contact Thomas Cayetanot at [email protected].