ReMiND Project: Midterm Evaluation Summary

Using mobile phone-based health technology (mHealth) in Uttar Pradesh (UP), India, the Reducing Maternal and Newborn Deaths Project (ReMiND) aims to increase the adoption of key maternal newborn and child health practices by improving the presentation and content of health information provided by Accredited Social Health Activists (ASHAs), as well as strengthening support and supervision structures for the ASHAs through the ASHA supervisor called the ASHA Sangini. (For more information about this project, which is a collaboration between Catholic Relief Services (CRS), Dimagi, and local partner Vatsalya, see Related Summaries, below.) This report summarises ReMIND's midterm evaluation.

The ReMiND project completed a baseline study in January 2013 that included a qualitative study and a quantitative knowledge, practice, and coverage (KPC) survey covering 1,100 households. The KPC established baseline measures for the project's strategic objective and targeted outcome-level indicators. Qualitative data were collected through semi-structured interviews with mothers, mothers-in-law, and fathers. Qualitative research encompassed practices, beliefs, and household decision-making surrounding prenatal care-seeking, delivery, postpartum care, and the wanted-ness of girl children. The midterm evaluation in August 2014 employed the same methodology with 2,200 households.

Key findings:

• One of the major successes of ReMind has been increasing ASHAs' coverage of pregnant women in their community. The project reduced the number of low-coverage ASHAs to less than 20% of the total, down from more than 60% at baseline. Only 44% of women received an ASHA visit with a counseling component prior to the intervention, but 72% received a counseling visit from their ASHA at the time of the midterm evaluation survey. Most of the improvement came from an increase in counseling on nutrition and rest during pregnancy, for which there was a module in CommCare. (ASHAs in the project use basic mobile phones operating Dimagi's open-source CommCare software, which equips them with multi-media job aids to support client assessment, counseling, and early identification, treatment and/or rapid referral of pregnancy, postpartum, and newborn complications, and tracking of childhood immunisations.
• ReMiND was particularly successful at reducing the number of low-coverage ASHAs: the number of ASHAs who counseled less than 40% of the pregnant women in their village dropped from 61% to 19%. ReMiND increased ASHA visits across all levels of women's education.
• As a result of the project, women received significantly more counseling on nutrition and pregnancy danger signs, and were able to name nearly twice as many pregnancy danger signs. Recall of nutrition messages increased from 17.8% to 55% of women, while recall of counseling danger signs increased from 3.8 % to 14% of women. Most of this increase came among less-educated women, such that after ReMiND, they named nearly as many danger signs as more-educated women. However, the absolute number of danger signs that they could name was still low (1.23). While knowledge of danger signs increased, care seeking for complications did not. Information about danger signs may have been difficult for women to act upon.
• The average woman accessed 41% more antenatal care (ANC) sessions at midterm than baseline, with 58% more women receiving the recommended number of three or more checkups (from 18.9% of women to 30.2%). The effects were concentrated among households with lower socioeconomic status, with much larger increases in ANC checkups among women with no or low education. However, 40% of women still did not receive any ANC at the time of the midterm evaluation survey.
• There was little significant change in other health outcomes, such as Iron and Folic Acid (IFA) consumption and institutional delivery. Findings demonstrated the importance of social networks, personal experiences, peer behaviours, and informal support systems in decision making about delivery. "In general, there are many constraints that prevent ASHAs from bringing in more women; targeting and eliminating these constraints must be a major focus of future work."
• Pride, honour, and community recognition that an ASHA gets in the community is a source of motivation for her. (ASHAs reported that some beneficiary families engaged their services primarily to view the multi-media content on their mobile phones; this was cited as an important motivating factor, a source of job satisfaction, and an indicator of their success.
• The evaluation found that use of mobile phones and the CommCare platform led to significant improvements in ASHAs' work by increasing:
  1. ASHAs' efficiency and productivity by compiling and organising beneficiary information and counseling content on the mobile phone instead of on paper registers, cutting down on paperwork and saving time;
  2. ASHAs' confidence in their own work, according to supervisors and administrators; and
  3. Community members' engagement with ASHA counseling, as well as their overall evaluations of ASHAs.

Recommendations are offered. For instance, the project has clearly demonstrated the importance of supervision and support for improved ASHA performance - findings that are reflected in the increase in ASHAs' coverage and that should be considered in future work. This supervision happens through two modalities: (i) The project's own supervisory staff, the Sector Facilitators, supporting the ASHAs; and (ii) the Sangini app that supports the government's own cadre of ASHA supervisors. According to organisers, friendly and supportive supervision serves many functions: it clarifies role responsibilities, demonstrates key skills (such as convincing reluctant families to go to a health facility), gives actionable feedback on performance, and helps solve problems and challenges in the field. Supportive supervision is mentioned frequently in government reports and recommendations, and yet the shift from an ethic of punitive "command and control" to collaboration and support is not easy. Having the Sangini, with the support of the app, in place, makes supportive supervision more visible and tangible.