Persuasive COVID-19 Vaccination Campaigns on Facebook and Nationwide Vaccination Coverage in Ukraine, India, and Pakistan

"A major current research gap is the lack of large, rigorously designed studies to more accurately determine the effectiveness of context-driven vaccine promotion messages disseminated through social media in promoting vaccine uptake..."

Vaccine misinformation spread on social media has not only undermined efforts of public health agencies and eroded trust but has also potentially increased vaccine hesitancy. At the same time, the reach and influence of social media could potentially be harnessed in a positive way to boost vaccine uptake. This study examined whether pre-tested, persuasive messaging campaigns from the United Nations Children's Fund (UNICEF), disseminated on Facebook, influenced COVID-19 vaccine uptake in Ukraine, India, and Pakistan.

Co-designed with UNICEF Headquarters and Country Offices in India, Pakistan, and Ukraine, the intervention comprised context-driven, evidence-based messages on Facebook, with the control condition receiving no messages. In each country, 5-8 ads were developed based on an analysis of data and insights from multiple sources: public posts analysis on Facebook (developed by Meta's Data for Good team), routine survey data from UNICEF, and a review of the peer-reviewed literature. These messages were then iteratively tested using Facebook's Brand Lift Surveys (BLS); results from this testing informed the final set of messaging interventions that were used for this study. All ads featured the UNICEF logo and contained a link to a website with more information on COVID-19 vaccination and a vaccination scheduler. The ad campaigns ran on Facebook and were optimised for reach to ensure maximum exposure to Facebook users 18–65+ years of age.

Briefly, the ad set in Ukraine comprised ads with practical information, safety and efficacy information, liberty-framed posts, and a social responsibility post. In India, the ad set tapped into national pride, safety of the vaccines and social responsibility. In Pakistan, the ads showed social responsibility and testimonials with vaccine accessibility information. Ads in India and Ukraine featured illustrations that were converted in GIFs to enhance user engagement; ads in Pakistan were framed as photographic testimonial. In each context, ads used a mix of content that was informed by Moral Foundations Theory, which emphasised values-based messaging to drive behaviour change (in Ukraine, ads focused on a liberty-frame; in Pakistan, ads focused on an equity frame, and in India, ads focused on national pride). Furthermore, ads featured trusted messengers in such contexts, which were identified in the BLS, although these differed by context (i.e., doctors and scientists were featured in India and Ukraine, alongside relatable family characters in Ukraine and Pakistan, to instill social responsibility for COVID-19 vaccines). Ads also emphasised safety and efficacy of vaccines and provided basic information on how to access vaccines.

To understand whether the intervention had an effect on COVID-19 vaccine uptake:

• In Ukraine, the researchers deployed a stepped-wedge randomised controlled trial (RCT). Half of the 24 oblasts (provinces) received five weeks of the intervention, the other half ten weeks of the intervention.
• In India, an RCT with an augmented synthetic control was conducted in five states (Bihar, Chhattisgarh, Jharkhand, Madhya Pradesh, Rajasthan), whereby 40 out of 174 districts were randomised to receive six weeks of intervention.
• In Pakistan, the researchers deployed a pre-post design, whereby 25 city districts received six weeks of the intervention.

Weekly COVID-19 vaccination data was sourced through government databases. Using Poisson regression models, the association between the intervention and vaccine uptake was estimated. In Ukraine, the researchers conducted a survey among Facebook users at three time points during the RCT, to ascertain vaccination intentions and trust in vaccines.

The campaigns reached more than 110 million Facebook users and garnered 2.9 million clicks. In Ukraine, the intervention did not affect oblast-level vaccination coverage (relative risk (RR): 0.93, 95% confidence interval (CI) 0.86-1.01). Similarly, in India and Pakistan, there was no effect of the intervention found (India: RR 0.85, 95% CI 0.70-1.04; Pakistan: RR 0.64, 95% CI 0.01-29.9). The survey among Facebook users in Ukraine showed that high trust in vaccination stakeholders was significantly associated with self-reported vaccine uptake for all information sources (e.g., UNICEF adjusted odds ratio (aOR): 5.09, 95% CI: 4.71-5.51). The inverse was also true; those with low trust in information stakeholders were less likely to be vaccinated against COVID-19 (e.g., government aOR: 0.21,95% CI 0.20-0.22).

Thus, the Facebook campaigns had a wide reach, which did not translate in shifting behaviours. Possible explanations for the observed null results include:

• It could be that "ads on Facebook alone are an insufficient vehicle to influence actual immunization coverage. Facebook usage varies strongly across countries, and in India and Pakistan it is dominated by men. Furthermore, while the metrics indicated that the ads were seen multiple times, this might not have been enough to create a lasting impression. Vaccine decision-making is inherently complex, and the sole influence of persuasive messaging delivered via online ads may not have been enough to affect behavior..."
• Given the timing of the study in early 2022, many people who were eager to get vaccinated may have already done so, and the intended audience may have shifted to include mostly people who were less intentioned to get vaccinated - a more difficult audience to convince.
• External events may have influenced COVID-19 vaccine uptake, including the looming threat of a Russian invasion in Ukraine and the large Omicron wave in all three countries that coincided with the intervention. For instance, the Omicron wave may have spurred the general public (i.e., in both intervention and control districts and oblasts) to get vaccinated, which may have biased an effect to the null.

In conclusion, the "results underscore that reach alone is not enough to shift behaviors. Gains might be made by using multi-pronged demand generation strategies, which include more customized messaging. Trust in vaccines and in sources of vaccination information was an important predictor of vaccination practices and should be leveraged in future vaccination campaigns."

Full list of authors, with institutional affiliations: Maike Winters, Yale University; Sarah Christie, Yale University; Chelsey Lepage, Irimi Company; Amyn A. Malik, Yale University; Scott Bokemper, Yale University; Surangani Abeyesekera, UNICEF Headquarters; Brian Boye, UNICEF Country Office India; Midhat Moini, UNICEF Country Office India; Zara Jamil, UNICEF Country Office Pakistan; Taha Tariq, UNICEF Country Office Pakistan; Tamara Beresh, UNICEF Country Office Ukraine; Ganna Kazymyrova, UNICEF Country Office Ukraine; Liudmyla Palamar, UNICEF Country Office Ukraine; Elliott Paintsil, Yale University; Alexandra Faller, The Public Good Projects; Andreea Seusan, The Public Good Projects; Erika Bonnevie, The Public Good Projects; Joe Smyser, The Public Good Projects; Kadeem Khan, Meta Platforms Inc.; Mohamed Gulaid, Meta Platforms Inc.; Sarah Francis, Team Upswell; Joshua L. Warren, University of Texas Southwestern; Angus Thomson, Irimi Company; Saad B. Omer, Yale University and University of Texas Southwestern