How successful is Facebook at removing COVID-19 disinformation?

Short answer: Not very.

Longer answer would be it probably depends on the country you're in and the language of the content.

We've decided to take a closer look on how Facebook removes COVID-19 related disinformation and hoaxes in Slovakia. Slovakia is a relatively small market for Facebook and it is publicly known that it only has a single fact-checker for the whole country, so we didn't expect to see much. But even with the bar set so low, the corporation managed to limbo dance under it somehow.

Dataset

Using Gerulata advanced filtering capabilities, we've selected something over 8 thousands of posts discussing COVID-19. All of these posts were published in February 2021 by pages active in slovak infosphere. Out of these, we've filtered 1249 posts created by pages that are, according to our systems, associated with disinformation content. These posts have a good chance of containing hoaxes or disinformation.

The selected posts were later manually labeled using out crowdsourcing platform Labelo, sorting the content into four categories:

• Obvious hoax or disinformation

• Likely hoax or disinformation

• Controversial opinion

• Everything else

The annotators didn't fact-check the posts, but instead relied on their own knowledge to categorise the posts. Here are some examples of content labeled as disinformation, translated from Slovak:

In hotspots of the "contagion" in Italy and Germany, NATO aircraft sprayed highly toxic nano-polymers causing COVID-19.

Three of the most diabolic entities on the planet were indicted by a Peruvian court a few days ago for conspiring to create a new coronavirus in order to achieve world domination. Bill Gates, George Soros and members of the Rockefeller family were found guilty of deliberately spreading the SARS-CoV-2 virus.

Covid vaccines pose a risk to all of Earth's inhabitants. Responsible people refuse to be vaccinated by genetic types of vaccines.

Methodology

In May 2021, we've looked at the disinformation posts from February and checked if they were deleted. In case they were deleted, we've also estimated when the deletion happen. We've also compared disinformation posts to other posts in terms of reactions and virality.

Findings

Our findings support the claim that disinformation content is in general more viral. We've also found that COVID-19 disinformation is more viral than other content from the same (disinformation) page.

Our data shows that Facebook is not very effective in removing COVID-19 disinformation from its platform. Not a single post from our sample was removed in February 2021.

Later, in March, Facebook removed one of the disinformation pages, Chemtrails Slovakia. This page was responsible for publishing 16 of the disinformation posts in our dataset.

While removing notorious publishers of disinformation content generally makes sense, such actions are in practice too sparse and too slow to make a real difference. A replacement page was immediately created, named "Chemtrails Slovakia+", and is already attracting followers and publishing the same toxic content as before.