Starting Conversations to Tackle Sanitation in India Through TV Drama: Evaluation of Navrangi Re!

"Impact was underpinned by high levels of narrative engagement and strong cognitive and affective responses to this engagement. The evaluation detected strong changes in attitudes, conversations and behavioural intent across the FSM continuum."

Entertainment-education - "edutainment" - interventions use storytelling in an effort to achieve social and behaviour change. There is a growing body of evidence indicating the effectiveness of edutainment interventions in low-resource settings that feature locally crafted and well-researched narratives to effect change. Building on this research, this study presents the results of a quasi-experimental evaluation of Navrangi Re! ("Nine to a shade"), a 26-episode television drama that aired in India in 2019. As described at Related Summaries, below, the drama aimed to influence sanitation behaviours by changing knowledge and attitudes, increasing risk perception, stimulating conversations, building collective efficacy, and creating social disapproval against poor faecal sludge management (FSM) practices. This article presents the results from the external evaluation of the impact of Navrangi Re!, undertaken by Oxford Policy Management and Centre for Social Behaviour Change (CSBC) in close collaboration with BBC Media Action.

After describing the drama, which reached 59.6 million unique viewers, the article outlines the challenge of applying rigorous and robust evaluation methods (e.g., randomisation) to edutainment interventions in real-world settings. In this case, a listing exercise was undertaken to identify households who watched the TV channel (at any time or day) that the show was going to be aired on (Rishtey). This approach made assumptions that not all households who watched the channel would end up watching the show , but that there would be a high degree of similarity between households who watched the channel and the show, and those who watched the channel but not the show. The evaluation compared changes in outcomes of those exposed to the TV show with those unexposed, applying differences-in-differences estimation to a panel of 2,959 respondents. Baseline balance tests showed high comparability between exposed and unexposed respondents.

The evaluation looked at:

• Proximal indicators related to narrative engagement of viewers with the show - Narrative engagement is understood as a multidimensional construct involving both cognitive and affective responses. The show was well received by viewers even at low levels of exposure (1+ episodes); they watched nearly complete episodes, and the vast majority reporting high levels of enjoyment, reactance, and emotional engagement. Narrative understanding was high for early episodes, but fell for later episodes, perhaps driven by the fall in viewership that occurred after a change in channel availability.
• Intermediate indicators related to the show's effects on knowledge, attitudes, social disapproval, and interpersonal communication - Table 5 in the paper shows the effect of exposure to Navrangi Re! on some key intermediate outcome indicators. For example, knowledge about the health impacts of faecal sludge improved, although the pervasive beliefs that bigger septic tanks are better (so that they are unlikely to overflow during the tenancy or lifetime of the respondent) did not see a significant change. Attitudes towards regular desludging, willingness to save to pay for this, and a desire to improve the quality of existing septic tanks saw significant improvements. The show was particularly effective at stimulating conversations between respondents and their family and friends on key issues.
• Distal indicators related to behavioural intent - The study found that exposure to the drama led to significant changes in most outcomes, with 37% of those who watched at least one episode showing behavioural intent to act, rising to 78% of those who had watched at least seven episodes.

In conclusion: "Overall, at exposure to over a quarter of episodes (7+ ) the intervention had a significant and positive effect on most outcome indicators. This shows that an edutainment intervention of this type can be successful in bringing about social and behavioural change on an invisible and a hard-to-address topic like FSM."