Advances in Theory Driven Design and Evaluation of Health Communication Campaigns: Closing the Gap in Practice and Theory

This chapter conceptualises a single communication campaign, applied to public health, as just one phase in a series of incrementally improved campaigns that eventually - if implemented and evaluated effectively - improve the health of a population. It centres around a case study of Scrutinize, a multimedia campaign for HIV prevention in South Africa. The description of this case follows a model of campaign design and evaluation that is theory based, so a new metatheory of health communication for campaign design and evaluation is also presented.

One figure in the opening sections of the chapter presents a 6-stage model of the communication design, implementation, and evaluation cycle. Because theory plays a role in all 6 stages, it is treated as the central core of the model. There is no real beginning to this cycle; previous communication and its evaluation, along with new formative research, should both be used to design a new campaign. Several iterations of this cycle are usually required to accomplish the desired health objectives.

A second figure presents a metatheory of health communication that summarises the contribution of a wide range of communication, social, and behavioural change theories and their interrelationships. The left-to-right order implies causal influences, as indicated by the direction of the arrows among the 4 stages (boxes) of the model and the underlying environmental constraints. From left to right, the model provides a descriptive, explanatory, and predictive model of change that can be applied to practice and to theory-based evaluation research. Communication design, however, should move from right to left, starting with the nature of the health issue and working backward to the changes in behaviour, ideation, and the communication approaches that are expected to have the desired effect. Meanwhile, the vertical, top-to-bottom order implies a social-ecological theory of communication - from the individuals to social networks to institutions to societal and then to environmental supports and constraints.

The Scrutinize case study illustrates how these processes work in practice. The first step was formative research, which began after the professional communication staff of Johns Hopkins Health Education in South Africa (JHHESA) studied the results of the national AIDS surveys in South Africa in 2005 and 2006. The Centre for AIDS Development, Research and Evaluation (CADRE), an HIV research agency in Johannesburg, then conducted 30 in-depth interviews and 6 focus group discussions (FGDs) to explore attitudes, norms, and practices related to multiple sexual partners (MSP). From the summary of the transcripts, 120 verbatim statements related to MSP were extracted, edited to improve clarity, and then pretested with 100 selected youth in Gauteng Province. A table in the chapter shows the set of 12 attitudinal statements that emerged from the factor analysis of these items. They offer insight into how South African youth think about MSP. Theory states that that they should predict MSP behaviour and that messages designed to change these attitudes would also change the behaviour.

The Related Summaries, below, provide information about the Scrutinize campaign, also detailed in this chapter's exploration of its creative insight and strategic design. In brief, the challenge for the new campaign was to find a way to get young men and women, ages 16 to 32 years of age, to pay attention to new HIV prevention messages that focused on 1) the risk of MSP, 2) the danger of failing to use condoms properly when drinking heavily, and 3) the importance of getting tested regularly for HIV. The design team partnered with animators, actors, and comedians to create an animated, consumer-oriented television campaign designed to convey energy, humour, excitement, and memorable messages. The brand name, Scrutinize, was proposed by the voice of the leading character as a way to remind people to look out for themselves and others. The primary slogan ("Eliminate the Element of Surprise - Scrutinize"), the campaign logo (an eye embedded within an AIDS red ribbon), and the payoff line ("Flip HIV to H.I. Victory") echoed the character's message of the need for the audience to look carefully at themselves and their partners' sexual behaviour and to reinforce the perceived efficacy of prevention behaviour. Two overlapping theories apply to these design decisions: the theory of reasoned action and planned behaviour and the extended parallel process model.

The animerts were pretested with FGDs and revised accordingly (YouTube versions of the final 7 animerts used in the campaign are available online here). A facilitator's guide for the community component was pretested with peer educators in several settings (see below). DramAidE trained peer educators and other partners to engage in their own entertainment-education (E–E) activities related to the campaign. Scrutinize also included promotional items such as bar coasters, stickers, umbrellas, hats, HIV risk cards, Levi's Scrutinize T-shirts, and posters that showed how to use condoms correctly, including one that illustrated sexual networks and linked the Scrutinize theme to football (soccer). Coverage of the campaign also appeared in newspapers.

From June 2008 to July 2009, the seven 40- to 60-second animerts were broadcast on all SABC television stations and in 370 health clinics nationwide. They were strategically placed in popular, prime-time television programmes with high viewership, such as TV dramas, sports, music, and youth reality shows. The broadcast schedule coincided with national events and holidays. The animerts were broadcast a total of 2,666 times, reaching 96% of South Africans aged 16 to 32 years who watched with an average frequency of 89.6 times. Scrutinize Online initiated a Facebook page where the audience could engage in key issues themselves and a Scrutinize website where the YouTube versions of the animerts could be seen. More than 1,500 young people friended the site on Facebook, and 2,000 joined an unofficial Scrutinize Fan Group on the web. The Scrutinize Live events reached out to communities, schools, universities, postsecondary education and training institutions, and out-of-school youth with its own E–E approach. DramAidE trained more than 637 peer educators in the Scrutinize methodology; an estimated 71,000 students were reached by means of interpersonal communication, promotion of HIV prevention, counseling, and HIV testing. ABC Ulwazi and DramAidE reached 640,000 people through campus radio and other media outreach. Activities were monitored by participating partners, who returned monthly reports on the number of individuals trained and reached.

A nationally representative sample survey of 9,728 men and women ages 16 to 55 years was conducted between June and August of 2009. Because the Scrutinize TV campaign was intended for youth 16 to 32 years of age who were sexually active, this subsample of 4,012 was used to measure the impact. The overall analysis showed that Scrutinize had statistically significant impacts on condom use, talking to one's friends and sexual partners about HIV testing, getting tested for HIV, and knowledge about the increased risk from having MSP. The case study presented here focuses only on the MSP objective; however, the short duration of the animerts did not allow time to address this MSP attitude, so the campaign should not be held accountable for any MSP attitude change. The campaign was also designed to increase knowledge of the risk of MSP. Respondents were asked how they could prevent HIV infection. Faithfulness (sticking to one partner) was mentioned spontaneously by 39.7%, and 13.7% mentioned reducing the number of sexual partners. The final measure was defined as mentioning either one or both of the two, which amounted to 46%.

As is explained here, a full-scale campaign is a field experiment without random assignment to treatment and control groups. To justify a causal inference in this situation, multiple causal attribution (MCA) analysis was used with the data from the national cross-sectional survey. MCA combines structural equation modeling (SEM), multiple probit regression, propensity score matching (PSM), and sensitivity analysis to estimate the net impact of the intervention. The chapter discusses the issue of causal attribution and shares findings, such as the fact that both men and women who understood the Undercover animert had a significantly higher knowledge of the risk of MSP. After satisfying the causal assumptions of SEM, PSM was used to construct a counterfactual condition to estimate the net impact of the animert. It was found that, among those who correctly recalled the animert, MSP was 14.7%, compared to 17.9% among those in the matched control group who did not recall it.

The authors speculate that the primary message of the Undercover animert - the 6-week acute infectious period for HIV - was new information for most of the audience. It may have elicited a renewed, heightened level of fear related to MSP. They suggest that future animert development should elaborate on this message, simplify it, and find a way to make it clear while still increasing the audience's confidence (self-efficacy) for dealing with it in their own lives.

The SEM uncovered an unexpected positive effect of the animert on MSP attitudes among a subset of the population: young men and those who think that they are not at risk of getting infected. That it occurred among young men who already have positive attitudes toward MSP supports the reactance theory. It is possible that showing entertaining animerts on television inadvertently legitimised and reinforced MSP attitudes among young men who already hold those attitudes and those who think they are not at risk. Heavy emphasis on MSP may have made it appear to be the social norm when, in fact, it is not. Perhaps the animert provoked an emotional, psychological reactance because of the perceived threat to one's personal sense of freedom. Because MSP attitude was not an explicit objective of Scrutinize, it is simply an unintended effect and one that needs to be addressed in future campaigns, according to the authors. Specifically, new animerts need to emphasise that MSP is not the social norm while still conveying the increased risk of infection during the acute infectious period.

The size of the population ages 16 to 32 years who had sex in the last 12 months was 10,784,684 (frequency weighted data). Of that population, 3,496,443 correctly interpreted the Undercover HIV animert (32.4%). Based on the PSM estimate, 3.2% of this group would have had MSP without exposure and correct understanding of the animert. This amounts to 111,886 young people. This animert was broadcast 107 times at a total cost of US$231,258. Thus, the cost-effectiveness of reducing MSP among those who would have had MSP without exposure was US$2.07 per person. However, almost half of the broadcast costs were paid for by SABC and e.tv. If this is taken into account, then the cost-effectiveness of the animert was US$1.09 per number of fewer cases of MSP.

In conclusion: "The cycle of campaign design and evaluation for the first Scrutinize campaign has identified how well and in what ways it affected the intended audience while at the same time providing new information to improve its effectiveness in the next cycle. Because the evaluation was theory driven as well as method driven, the results have also made contributions to the body of knowledge in several fields of scientific inquiry as well as to professional practice."