Evidence-based Public Health: Not Only Whether It Works, but How It Can Be Made to Work Practicably at Scale

"Public health decision-making requires knowledge of not just whether something works under particular circumstances but also how, when, and why for broad application."

The core argument of this editorial is that evidence-based public health (EBPH) must go beyond reliance on randomised controlled trial (RCTs) to include other methodologies to arrive at optimal public health programming. In it, the author, Dr. James Shelton, explains that RCTs typically zero in on very specific issues in constrained time and place, which can work for medical questions about individuals but not in the public health context - where the goal is to address the needs of large populations across clinical, behavioural, and structural platforms. Thus, because situations can vary so widely, "external validity" (generalisability of evidence to other situations) is "absolutely crucial for public health applications. Yet external validity is a severe weakness of the RCT methodology for public health decision-making." Furthermore, according to Dr. Shelton, many agencies like the World Health Organization (WHO) are committed to ending preventable infant and maternal death - an objective requiring a range of interventions, such as immunisation, antibiotics for pneumonia, and promotion of exclusive breastfeeding.

Dr. Shelton provides several examples of the problematic application of RCTs to public health. For instance, a "cluster" randomised trial (these RCTs involve randomising population groups rather than individuals) involving peer mentors with HIV infection to promote wellness behaviour among pregnant South African women living with HIV (WLH) drew this conclusion: "WLH benefit by support from HIV-positive peer mentors..." But, Dr. Shelton disputes that, noting that: the study was small and localised to one area of South Africa; it found significant and mostly modest change in only 4 of 19 behaviours; and it "provided little intervention detail and no qualitative evidence about the thinking and motivations of the WLH."

So, what is the alternative? Dr. Shelton's "theory of change" to inform EBPH is based on the idea that validity often derives from whether things are successful in a particular environment. "Broader applicability emerges when consistent patterns of findings or collective 'lessons learned' materialize." For example:

• Positive deviance (PD) - involves identifying a repeated pattern of success seen across many different situations, providing confidence in the general approach. For instance, the non-governmental organisation Marie Stopes provided more than 700,000 contraceptive implants in 2012 across a variety of countries in sub-Saharan Africa. In a research paper that Dr. Shelton cites, Marie Stopes describes its 3 service delivery modalities, along with operational details including provider training, client outreach, robust supply chains, and quality assurance measures.
• Systematic trials coupled with programme testing - involves a variety of methodologies. For instance, Dr. Shelton cites an analysis of 12 successful community-based child survival projects that found that intensive outreach to caregivers and community leaders was a crucial common element.
• Performance improvement - involves managers (typically in collaboration with staff) assessing critical strengths and weaknesses in programmes. They formulate solutions, test them, and measure whether and how performance improves. As in the PD approach (outlined above), generalised knowledge can arise when patterns of solutions emerge, common across multiple programme experiences - as has happened in research on male circumcision, a paper Dr. Shelton cites here.
• Simple measurement across the causal pathway - involves a variety of data to assess and guide implementation. For instance, Dr. Shelton references a paper describing a worldwide analysis based on Demographic and Health Surveys (DHS) data, which revealed that a substantial proportion of people access the private sector for key child health services in many developing countries. This finding supported the arguement for more programmatic effort to engage private-sector providers.
• Additional epidemiologic methods, including cohort and case-control studies.
• Modeling, which explores the implications of data, such as whether antiretroviral drugs (ARVs) not only reduce HIV transmission but actually abate the HIV epidemic at the population level.
• Human functionality, culture, and biology - involves experimental and observational methods to analyse issues such as how social networks influence behaviour or which neuroimaging changes correlate with approval or disapproval when individuals see an anti-smoking advertisement.
• Summative evaluations, which assess programme effectiveness, making use of a variety of methods as described above.

In closing, Dr. Shelton reiterates the importance, within EBPH, of identifying and synthesising patterns of findings across multiple experiences in enough detail to meaningfully inform similar efforts across a variety of situations. Along the lines of the PD approach described above, he writes: "Success in programs, including scaling up, often depends on finding and cultivating committed and capable 'champions.'" He also observes that population-level behavior change, such as reducing tobacco use, "most often results not from any one single campaign or intervention but from a sustained combination of interventions, including structural interventions such as increasing taxation, individual persuasion, and changing social norms."

Going forward: "A major agenda for EBPH is identifying such common patterns and helping program managers adapt and apply that knowledge."