How Facebook Failed To Prevent Stop The Steal

Last month, Facebook CEO Mark Zuckerberg testified in front of a House of Representatives committee that his company had done its part “to secure the integrity of the election.” While the social network did not catch everything, the billionaire chief executive said, Facebook had “made our services inhospitable to those who might do harm” in the lead-up to the Jan. 6 insurrection.

Less than a week after his appearance, however, an internal company report reached a far different conclusion: Facebook failed to stop a highly influential movement from using its platform to delegitimize the election, encourage violence, and help incite the Capitol riot.

Shared on Facebook’s employee communication platform last month, the report is a blunt assessment of how people connected to “Stop the Steal,” a far-right movement based on the conspiracy theory that former president Donald Trump won the 2020 US presidential election, used the social network to foment an attempted coup. The document explicitly states that Facebook activity from people connected to Stop the Steal and other Trump loyalist groups including the Patriot Party played a role in the events of Jan. 6, and that the company’s emphasis on rooting out fake accounts and “inauthentic behavior” held it back from taking preemptive action when real people were involved.

“Hindsight is 20/20, at the time, it was very difficult to know whether what we were seeing was a coordinated effort to delegitimize the election, or whether it was free expression by users who were afraid and confused and deserved our empathy,” reads the report, which was put together by an internal task force studying harmful networks and obtained by BuzzFeed News. “But hindsight being 20/20 makes it all the more important to look back to learn what we can about the growth of the election delegitimizing movements that grew, spread conspiracy, and helped incite the Capitol insurrection.”

The report, titled “Stop the Steal and Patriot Party: The Growth and Mitigation of an Adversarial Harmful Movement,” provides yet another case study of how relatively small but coordinated groups of people are able to wreak havoc and spread misinformation on the world’s dominant social network. It’s also a sober admission that a company, which recorded more than $29 billion in profit last year, still struggles to track and preempt networks of people seeking to sow discord and undermine liberal democracy in the US and around the world.

Even though the company spent months preparing for potential delegitimization of the election from Trump and his supporters, Facebook was outmaneuvered by a powerful network of coordinated accounts that promoted groups where members glorified hate, incited violence, and sought to spread a big election lie, according to the report. It notes that while Facebook was satisfied “at having made it past the election without major incident,” that feeling was “tempered by the rise in angry vitriol and a slew of conspiracy theories that began to steadily grow” after Election Day, Nov. 3.

Joan Donovan, research director of Harvard University’s Shorenstein Center on Media, Politics, and Public Policy, said the Stop the Steal movement began long before Election Day, and Facebook’s failure to prepare shows it’s unable to protect democracy.

“For me, at the end of the day, it comes down to: Do you care? Do you care enough about democracy? Do you care enough about the fate of the nation to ensure that your product is not used to coordinate and overthrow the government?” she said. “There is something about the way Facebook organizes groups that leads to massive public events. And when they’re organized on the basis of misinformation, hate, incitement, and harassment, we get very violent outcomes.”

In response to questions for this story, a Facebook spokesperson said the company took a number of steps to limit content that sought to delegitimize the election, including the suspension of Trump’s account, labeling candidates’ posts with vote-counting information after Trump prematurely declared victory, and the removal of the original Stop the Steal group.

“As we’ve said previously, we still saw problematic content on our platform during this period and we know that we didn’t catch everything,” Facebook said in a statement. “This is not a definitive report. It’s a product of one of many teams who are continuing to study what happened so we can continue improving our content moderation.”

Employees were made aware of the original Stop the Steal Facebook group, which emerged on election night, after it was “flagged for escalation because it contained high levels of hate and violence and incitement (VNI) in the comments.” By the time Facebook removed it, on Nov. 5, it had become a movement, amassing more than 300,000 members in a 24-hour span with more than a million people wanting to join. The group’s takedown and splintering into offshoot groups caused a major problem for Facebook, which took a “piecemeal” approach to enforcement and failed to see Stop the Steal as a wider, harmful movement, according to the internal report.

“Because we were looking at each entity individually, rather than as a cohesive movement, we were only able to take down individual Groups and Pages once they exceeded a violation threshold,” the report reads. “After the Capitol Insurrection and a wave of Storm the Capitol events across the country, we realized that the individual delegitimizing Groups, Pages and slogans did constitute a cohesive movement.”

Do you work at Facebook or another technology company? We’d love to hear from you. Reach out to ryan.mac@buzzfeed.com or via one of our tip line channels.

It was only after the violence of Jan. 6, according to the report, that Facebook teams realized they were dealing with a movement that “normalized delegitimization and hate in a way that resulted in offline harm and harm to the norms underpinning democracy.” And while the company spent months preparing for people to dispute election results, the report calls delegitimization a “new territory” in which “few policies or knowledge existed” prior to election night.

The document contradicts Zuckerberg’s statement to Congress about Facebook being “inhospitable” to harmful content about the election, and refutes chief operating officer Sheryl Sandberg’s January comment that the insurrection was “largely organized on platforms that don’t have our abilities to stop hate, don’t have our standards and don’t have our transparency.” It also shows that while Facebook said it was prepared for election destabilization and was monitoring signals for unrest, it failed to stop a movement that led to real-world harm.

Facebook disputed the idea that the report went against Zuckerberg’s and Sandberg’s public statements and noted that both had said there was violative content on the platform that the company did not catch.

Donovan said the report’s revelations and the misleading comments by Facebook leadership expose the true nature of the company and its products.

“This shows the company is anti-democratic at the very least,” she said, “and at the very worst, it shows that they know the risks, and they know the harm that can be caused and they are not willing to do anything significant to stop it from happening again.”

The report focuses on two actors in the election delegitimization movement on Facebook: Stop the Steal, which started as a fast-growing Facebook group before splintering into various offshoots, and the Patriot Party, an attempt led by far-right personalities to create a new political party for Donald Trump.

Created on the night of Nov. 3, the original Stop the Steal Facebook group was started by pro-Trump activist Kylie Jane Kremer, who said at public events leading up to Jan. 6 that it had more than a million people waiting to be added before it was shut down by the company two days later. The removal of the original group had the effect of cutting off the head of a hydra, as copycat and offshoot groups sprung up in its place.

The original Stop the Steal group, and the offshoots that emerged after it was banned, grew quickly thanks to what the report labels “super-inviter” accounts. The biggest Stop the Steal groups had 137 super-inviters, who invited 67% of the groups’ members, according to the report. These accounts were each responsible for inviting more than 500 people to groups. Facebook’s analysis found the super-inviters worked in coordination, lied about their locations, and used private groups and chats to coordinate activity.

The report compared this behavior to “growth hacking,” a marketing term often used to describe using shortcuts or other tactics to boost the popularity of a brand or business.

“Growth hacking may not always be bad,” the report reads. “A democratic movement, a movement seeking human rights, or an advertising movement, may all employ legitimate techniques to grow their audience quickly. However, when the growth is mixed with the signals of harm we described above, this rapid growth indicates the spread of harm, and may indicate coordinated harm.”

Facebook declined to comment on what, if any, action it took against super-inviter accounts.

“From the earliest Groups, we saw high levels of Hate, [violence and incitement] and delegitimization, combined with meteoric growth rates — almost all of the fastest growing FB Groups were Stop the Steal during their peak growth,” the report reads.

A table in the report illustrates how much hate existed in Stop the Steal groups compared to other political and social issue groups. While 0.13% of all civic groups on Facebook had at least five comments with white supremacist terms, more than 4% of Stop the Steal groups exhibited the same characteristic. Similarly, while 0.89% of civic groups on Facebook had at least five pieces of content deemed as “violence and incitement,” nearly 43% of Stop the Steal groups showed the same behavior.

The report also revealed that people affiliated with hate and right-wing militant organizations were active in Stop the Steal and Patriot Party groups.

“One of the most effective and compelling things we did was to look for overlaps in the observed networks with militias and hate orgs,” the report says. “This worked because we were in a context where we had these networks well mapped.”

The groups included people belonging to the Proud Boys, whose groups and pages had been banned by Facebook in 2018 and whose members were indicted for their role in the insurrection.

In a lengthy post to his own Facebook page last September, Zuckerberg outlined the steps his company had taken to protect the election, including the strengthening of “enforcement against militias, conspiracy networks like QAnon, and other groups that could be used to organize violence or civil unrest in the period after the elections.” Facebook’s own researchers, however, found that members of the supposedly banned groups remained on Facebook and were able to link up with Stop the Steal and Patriot Party supporters to help undermine the election.

BuzzFeed News and researchers have previously shown how militant and extremist groups and pages continued to operate and recruit on the social network, long after the bans were put in place.

On Dec. 31, for example, an image superimposed with the text “OCCUPY CONGRESS THE GREAT BETRAYAL IS OVER” was shared to a private Patriot Party group, which had about 23,500 members.

“If they won’t hear us, they will fear us,” read another piece of text on the image, which included the date, Jan. 6, 2021.

The report says Stop the Steal was “flagged for escalation because it contained high levels of hate and violence and incitement (VNI) in the comments.” But by the time it was removed, it had already become central to a burgeoning movement.

“It wasn’t until later that it became clear just how much of a focal point the catchphrase would be, and they would serve as a rallying point around which a movement of violent election delegitimization could coalesce.”

The phrase “Stop the Steal” predates the Facebook group of the same name started by Kremer, and was used by Trump adviser Roger Stone back in 2016. It reemerged as a rallying cry for bogus claims of election fraud among Trump supporters in the months before the 2020 election, a fact that Facebook should have been well aware of, according to Donovan.

“Everybody knew that Stop the Steal or something like it was on its way,” she said. “That went on for months prior to Jan. 6. It’s not like it was hiding behind the kind of tricks that Russia were using in 2016. They’re not hiding behind any of that.”

Donovan said the company’s failure is partly a result of its focus on inauthentic activity, such as people using fake accounts. Facebook ignores how its products create coordinated activity among real people, and the harm that can result, she said.

“They have insisted that somehow inauthenticity is the measure by which they should take action,” Donovan said. “This report is not path-breaking if you’ve been studying the way in which disinformation operators have evolved. In 2016, you had to engineer lots of fake engagement and stories because the networks were not mature enough. It’s only after you have four years of MAGA and the Trump caravan and the anti-vaxxers meeting up with the militia groups during the pandemic that you start to see these networks become agile, extensible, and adaptable to the moment.”

Facebook’s own report found that Kremer and other far-right figures were able to use Facebook to grow their influence in the period between the election and the insurrection.

After Facebook banned Kremer’s Stop the Steal group on Nov. 5, it became a rallying cry for her and her mother, Amy Kremer, during their subsequent cross-country bus tour, where a BuzzFeed News analysis found that speakers spread false lies about the election and at times explicitly advocated for violence. The tour was led by their organization, Women for America First, which also obtained the permit for the Washington, DC, event on Jan. 6 where Trump called upon supporters to march to the Capitol.

The Kremers did not respond to requests for comment sent to Women for America First.

“It has 1.2 million people in the queue [waiting to be added]. Well, what do you think happened? Facebook shut it down,” Amy Kremer said at a Morehead City, North Carolina, rally on Dec. 2 where another speaker joked about conservatives shooting liberals.

“It came all the way from the top from Mark Zuckerberg to shut it down, so they shut it down,” Kylie Jane Kremer said at a rally in Des Moines, Iowa, on Dec. 6.

Like the Kremers, Ali Alexander, a far-right activist who has spread conspiracy theories about Rep. Ilhan Omar and Vice President Kamala Harris, worked to undermine the election offline and used Facebook to supercharge the Stop the Steal movement. He appeared at the Jan. 6 rally and also worked with GOP lawmakers to put “maximum pressure on Congress while they were voting,” according to the Washington Post.

Facebook’s analysis found that he and the Kremers were key figures in growing the movement and said Alexander, an early promoter of “Stop the Steal” as a slogan, “was able to elude detection and enforcement with careful selection of words, and by relying on disappearing stories,” an ephemeral content feature in which images or posts disappear after 24 hours.

“We also observed [Alexander] formally organizing with others to spread the term [“Stop the Steal”], including with other users who had ties to militias,” the report reads. Alexander did not respond to a request for comment.

Online, people associated with the pro-Trump Patriot Party attempted to recruit people to their groups from Stop the Steal, according to Facebook’s findings. As a result, there was overlap between the networks of people who joined their respective groups. The report found that the content and invitations to these groups were linked to “higher levels of hate and violence, suggesting that these movements were harmful and that the harm was perpetuated through a network.”

It concludes Stop the Steal and Patriot Party “normalized delegitimization and hate in a way that resulted in offline harm and harm to the norms underpinning democracy.”

Facebook’s researchers also outline the bureaucratic, policy, and enforcement struggles of the social giant when trying to respond to a coordinated, fast-paced movement that exploits its platform to spread hate and incite violence. Despite the company removing the most populous Stop the Steal groups from its platform, the enforcement was “piecemeal” and allowed other groups to flourish.

The company admitted that it only realized it was a cohesive movement “after the Capitol Insurrection and a wave of Storm the Capitol events across the country.”

Facebook suspended Trump’s account late on the evening of Jan. 6. By then it had spent months trying to remove Stop the Steal groups, but was still failing.

A Jan. 8 BuzzFeed News analysis found 66 active Stop the Steal groups and 72 events. The largest group was private, touted 14,000 members, and explicitly said its goal was “to make aware the issues of fraudulent voting practices and Fraudulent ballot counting. also, to make these issues transparent for all!”

The report identifies ways for Facebook to improve its systems and detection, and raises questions and about gaps that its current policies don’t address.

“What do we do when a movement is authentic, coordinated through grassroots or authentic means, but is inherently harmful and violates the spirit of our policy?” the report says. “What do we do when that authentic movement espouses hate or delegitimizes free elections?”

Ultimately, the report says, the issue is that the company is not prepared to deal with what it calls “coordinated authentic harm.”

“We learned a lot from these cases,” the report says. “We’re building tools and protocols and having policy discussions to help us do better next time.”

The report echoes previous high-profile examples where Facebook failed to act and later issued a report promising to do better. In 2017, a postmortem concluded Facebook failed to combat disinformation, fake accounts, and other efforts to manipulate discourse around the 2016 election. In 2018, a human rights report commissioned by the company concluded that it failed to stop its platform “from being used to foment division and incite offline violence” in Myanmar, helping fuel an alleged genocide.

Donovan said the networks of people that made Stop the Steal a powerful force on Facebook are not going away, and things could get even worse given the company’s history of failure combating harmful movements.

“Unfortunately, in the long run, what we know about social media is that people don’t relinquish these networks if the companies do nothing to stop this coordinated infrastructure from reassembling itself like some crazy Voltron,” she said. “We’re going to be fighting the same thing in 2022 and 2024. And it’s only going to get bigger, badder, more messy, more agile, more flexible — and more violent.” ●