Methods for Social Media Monitoring Related to Vaccination: Systematic Scoping Review

"[S]ocial media monitoring opens the door to more dynamic research that continuously evolves and responds to a perpetually changing world."

Amplification and global spread on social media of public concerns about vaccination can diminish public trust in information provided by authorities and experts, negatively influencing vaccine decisions. Social media monitoring provides opportunities for authorities to listen, in real time, to online narratives about vaccines, and to detect changes in sentiments and confidence early. The aim of this study was to identify "infoveillance" methods used for monitoring vaccination-related topics on different social media platforms and to assess their effectiveness and limitations. The findings presented in this paper come from a broader European Centre for Disease Prevention and Control (ECDC) technical report (see Related Summaries, below). The aim of the ECDC report was to provide guidance for public health agencies to monitor and engage with social media, whereas this paper primarily focuses on the academic implications of social media monitoring.

For the purpose of this scoping review, the researchers conceived of social media as not simply a means of communication but, further, as a space within which individuals socialise and organise. The review therefore focuses on social networking sites and content communities, and it excludes online platforms that do not have social interactions as their main purpose (e.g., blogs or websites with a comments section). With that frame in mind, in December 2018, the researchers applied a comprehensive search strategy to multiple databases, resulting in 86 articles on social media monitoring of vaccination.

The first study identified on social media monitoring around vaccination was published in 2006, with an increasing number of studies published yearly since then. Most studies analysed online discourse on Twitter (n=42), YouTube (n=12), Facebook (n=11), and online forums (n=9). (See the portion of the paper titled "Accessing Data From Different Social Media Platforms: The Twitter Bias" for more on this issue.) A diversification of social media platforms can be observed in recent years, with studies of platforms such as Pinterest (n=1), Weibo (n=1) [100], Reddit (n=1), or Yahoo! Answers (n=2) all published after 2015. Seven studies monitored a mix of social media platforms.

The researchers developed a 3-step model of social media monitoring, organising the findings according to these phases:

1. Preparation: The main objective of the majority of studies in this review (55/86, 64%) was to better understand how vaccination is portrayed on social media, whether through the analysis of online discourse or sentiments, or by looking at how information is produced, shared, and engaged with. Many studies (15/86, 17%) used social media monitoring as a way to better understand general public discussions on vaccination, assuming that online discussions are a good proxy for vaccine confidence in a country or region.
2. Data extraction: Although 35 out of the 86 studies used manual browser search tools to collect data from social media, this was time-consuming and only allowed for the analysis of small samples compared to social media application programme interfaces (APIs) or automated monitoring tools. Only 10 out of the 86 studies used comprehensive lists of keywords (e.g., with hashtags or words related to specific events or concerns).
3. Data analysis: Partly due to privacy settings, geolocalisation of data was difficult to obtain, limiting the possibility of performing country-specific analyses. Once data from social media were extracted from studies, different analyses were performed. Twenty of the 86 studies performed trend or content analyses, whereas most of the studies (70%, 60/86) analysed sentiments toward vaccination. (See the portion of the paper titled "Social Media Analyses: Complexity of Analyzing Sentiments" for more on this topic.) Finally, 49 out of the 86 studies determined the reach of social media posts by looking at numbers of followers and engagement (e.g., retweets, shares, likes).

The researchers suggest that this model "was found to be useful in structuring methodologies for social media monitoring, and could be used in the future as a standardized protocol for performing social media monitoring. Further research could be performed to evaluate different components of the model, and propose a more detailed and complex framework for media monitoring."

The discussion section of the paper explores some implications of the findings. For example, the paper revealed the wide range of methods that are being used in the field of social media monitoring as a means to research and understand public sentiments around vaccination. Notably, "almost none of the articles evaluated the precision and accuracy of their monitoring and analysis methodologies. Furthermore, researchers have not drawn on a coherent body of agreed-upon methodologies....However, it may be apt that social media monitoring remains a flexible research design, as the nature and access to social media discourse on vaccination is continuously evolving."

In conclusion: "Future research should focus on evaluating these methods to offer more evidence and support the development of social media monitoring as a valuable research design."