An Empirical Comparison of Technologically Mediated Advertising in Under-connected Populations

"[V]oice-based entertainment services accessible over simple mobile phones can be used as vehicles for spreading development services..."

In developing regions, technology-based interventions typically require significant resources to achieve scale beyond the pilot stage. Spreading awareness, acquiring users, and retaining them over time are all significant barriers to scale. Interactive voice response (IVR) is a tool for ensuring more inclusion of people who do not have access to any other forms of technology beyond simple mobile phones and who are not literate enough to consume and produce textual forms of information. This paper compares seven advertising channels, including one IVR entertainment service, to acquire and engage users for - that is, to scale up - Super Abbu (Super Dad), a hotline designed to promote maternal-health awareness among low-literate men in Pakistan.

Super Abbu allows expectant fathers to ask questions and get answers from doctors, listen to publicly posted questions and (doctors') answers, and share relevant personal stories with peers. To interact with Super Abbu, users place a missed call, and the service calls them back. Super Abbu is a public service, and does not require users to login or register. However, all user-generated questions and stories are moderated to prevent the spread of inaccurate, irrelevant, and inappropriate information, while maintaining the decorum of the voice community. The service's volunteer doctors, who are recruited from local public and private hospitals, listen and respond to user questions via a smartphone app by recording an audio or a text message. All responses are re-recorded by a female speech artist as "Dr. Saba", a female doctor's persona created for Super Abbu. Users receive these responses in their next interaction with the service. Questions marked public by the users are posted publicly along with Dr. Saba's responses.

The IVR service is specifically meant for low-literate expectant fathers, as pregnancy and childbirth remain taboo topics among men in Pakistan - a country where maternal and child health indicators remain poor. Traditional maternal health information services for mothers often fail to achieve desired outcomes, as women are not the primary decision makers in most Pakistani households. Pakistan is a patriarchal society in which men also have more access to technology. With an understanding of the local context and with the aim to design beyond the user, Super Abbu provides a father with information about the health and well-being of his wife (and child), with the expectation that the wife will be the ultimate beneficiary.

Following pilot deployment, it was decided to scale Super Abbu to 20,000 users. Organisers explored all available options for reaching the desired number of users in the intended demographic: low-income, low-literate men of reproductive age. Being constrained by both budget and time, it was important that the advertising channels reach a large number of people rapidly, and at a low cost per user. Based on a literature review and knowledge of the local culture, organisers identified seven channels: flyers, banners displayed at the back of auto rickshaws, video ads on local cable TV, targeted video ads on Facebook, audio ads on local radio, robocalls, and audio ads on a popular entertainment-driven IVR service, "Polly". The latter is designed to reach low-literate, low-income populations via a simple voice-based game. When users call Polly, they are able to record and send voice messages to their friends in morphed, funny voices, making Polly playful while also providing a utility of being able to send free voice messages.

The paper presents results from an 11-week campaign where organisers acquired users for Super Abbu through the seven aforementioned advertising channels. Across these channels, they eventually reached 21,770 users, who engaged in 32,625 interactions on Super Abbu. To assess the efficacy of the channels, organisers considered three main user acquisition metrics: conversion rate, cost of user acquisition, and retention rate. Furthermore, to understand whether the IVR users (users of Polly) interact with Super Abbu differently from users acquired through other channels, the researchers compared users in terms of their activity, engagement, and IVR use sophistication.

Overall, the results show that the IVR-based entertainment service (Polly) outperformed other advertising channels on all considered user acquisition metrics with the exception of robocalls, which led in terms of spread. One of the key findings is the high conversion rates (the fraction of people reached through an advertisement channel who end up calling Super Abbu) for Polly (50%) and robocalls (25%) compared to other channels (less than 0.1%). The researchers explain this finding by pointing to Polly's entertainment value, which motivates users to press a button and explore the advertised service (Super Abbu). Also, both robocalls and Polly employ the same modality (IVR) as Super Abbu. The IVR advertisement services may offer a smoother transition in terms of pre-training or at least priming the users to navigate speech interfaces before transitioning to Super Abbu. Yet another reason for the high conversion rate of robocalls could be that these calls were made to users of government-run information services (health and agriculture), so they may be primed and pre-selected to be more open to such services, given they had volunteered their phone numbers.

Users acquired through the IVR entertainment service (Polly) also performed better than other channels in all interface-related measures (activity, feature engagement, use of sophisticated interface features, and retention). Polly offered the lowest cost per retained users of Super Abbu, and most users outside of Polly did not end up becoming long-term users of the service.

The researchers conclude: "While our project scaled a voice-based maternal-health hotline using an IVR-based entertainment platform, our results demonstrate more generally that compared to traditional channels of spread, entertainment has a great potential to be leveraged for acquiring users for development services and may even help resolve the critical bottleneck of large-scale impact in HCI4D [human-computer interaction for development] interventions."