Spatiotemporal Trends in COVID-19 Vaccine Sentiments on a Social Media Platform and Correlations with Reported Vaccine Coverage

"Monitoring data on social media through social media listening can play a crucial role in assisting policy-makers."

Social media listening enables researchers and policymakers to understand the ever-changing dynamics of the public's response to public health measures during a pandemic. This study fine-tuned multilingual deep learning models to analyse posts on X (formerly called tweets) on COVID-19 vaccination in 90 languages that were made between late 2020 and early 2022. The researchers assessed global geographical and temporal trends in the acceptance of COVID-19 vaccines among platform users from 135 countries and territories and validated findings using statistical data on COVID-19 vaccination coverage. They also explored the determinants of trends in COVID-19 vaccine acceptance among platform users.

The researchers collected over 13 million posts on the social media platform X (formerly Twitter) regarding COVID-19 vaccination made between November 2020 and March 2022 (between the emergency approval of COVID-19 vaccines and the time when over half of the world's population had been vaccinated) in 90 languages. Multilingual deep learning XLM-RoBERTa models annotated all posts using an annotation framework after being fine-tuned on 8,125 manually annotated, English-language posts. The annotation framework covered 4 key concepts related to COVID-19 vaccination and included 8 categories. First, COVID-19 vaccine acceptance covered the categories of: (i) intent to accept vaccination; and (ii) intent to refuse vaccination. Second, confidence in COVID-19 vaccines covered: (iii) belief that vaccines are effective; (iv) belief that vaccines are not safe; and (v) distrust in government. Third, the online information environment regarding COVID-19 vaccines covered: (vi) misinformation or rumours about vaccines. Fourth, perceived barriers to accessing COVID-19 vaccines covered: (vii) vaccine accessibility; and (viii) vaccine equity.

The annotation results were used to assess spatiotemporal trends in COVID-19 vaccine acceptance and confidence as expressed by platform users in 135 countries and territories. The researchers identified associations between spatiotemporal trends in vaccine acceptance and country-level characteristics and public policies by using univariate and multivariate regression analysis.

The study found that acceptance of, and confidence in, COVID-19 vaccines varied considerably across World Health Organization (WHO) regions: The proportion of platform users who expressed COVID-19 vaccine acceptance throughout the study period varied from 33.2% to 78.1% across countries and territories, and the proportion who expressed an intention to refuse vaccination varied from 5.9% to 24.9%. A greater proportion of platform users in the WHO's South-East Asia, Eastern Mediterranean, and Western Pacific regions expressed vaccine acceptance than users in the rest of the world. Platform users in the African Region and the South-East Asia Region more often posted on vaccine accessibility and vaccine equity than users elsewhere.

Countries in which a greater proportion of platform users expressed vaccine acceptance had higher COVID-19 vaccine coverage rates. Trust in government was also associated with greater vaccine acceptance. Country-level characteristics that had a significant negative association with vaccine refusal included better governance, pandemic preparedness, trust in government, and the level of social development.

Internationally, vaccine acceptance and confidence declined among platform users as: (i) vaccination eligibility was extended to adolescents; (ii) vaccine supplies became sufficient; (iii) nonpharmaceutical interventions were relaxed; and (iv) global reports on adverse events following vaccination appeared. The proportion of users who posted about misinformation or rumours on COVID-19 vaccination generally increased during the observation period. "Recognizing public fears and their origins is the first step in devising a rapid educational response." In light of these findings, the researchers urge policymakers to proactively prepare to increase public support for vaccination in future pandemics, in addition to implementing public health surveillance.

Other suggestions offered revolve around the study's finding of the importance of trust in boosting vaccine acceptance and coverage, which is consistent with previous research. Consequently, building trust in government should be a priority for policymakers seeking to promote compliance with public health interventions, including vaccination.

In terms of methodology, the researchers argue that social media listening based on multilingual deep learning models has several advantages:

• Unlike traditional research methods such as surveys, it can rapidly and thoroughly scan the whole dynamic information environment for digital opinions derived from public contributions and interactions, without researcher involvement.
• At is not affected by the reporting bias that can result from interactions with researchers, social media listening can be particularly useful for research on sensitive public health issues.
• It could provide real-time insights into public sentiment to inform public health interventions, especially during outbreaks and pandemics.
• It is cost-effective and could be applied in low-resource settings.

Nevertheless, social media listening faces challenges, such as:

• The potential non-representativeness of social media data (e.g., social media users are typically skewed towards younger individuals, who may be over-represented in anti-vaccine groups);
• Susceptibility to short-term noise (i.e., random fluctuations in opinion);
• A lack of demographic information; and
• A reliance on manually annotated data.

That said, the researchers propose several avenues to address such challenges, including the application of social media listening to social media platforms other than X, such as Facebook, Reddit and Instagram, which may be used by hard-to-reach population groups.

In summary, social media listening using machine learning can address complex public health issues across diverse settings and in many languages. "Social media listening could provide an effective and expeditious means of informing public health policies during pandemics, and could supplement existing public health surveillance approaches in addressing global health issues."