Assessing Sustainment of Health Worker Outcomes beyond Program End: Evaluation Results from an Infant and Young Child Feeding Intervention in Bangladesh

"Sustaining components of effective interventions and program activities is essential for maintaining and supporting improvements in IYCF [infant and young child feeding] practices."

Alive & Thrive (A&T) supported interventions to improve maternal nutrition and infant and young child feeding (IYCF) practices in several countries including Burkina Faso, Ethiopia, India, Nigeria, and Vietnam. In Bangladesh, A&T was implemented from 2009 to 2014. This study asks: Is there evidence of sustained differences in outcomes (quality of IYCF service delivery, IYCF knowledge, job satisfaction, and job readiness) among health workers in intervention areas, vs. those in comparison areas, after the A&T initiative ended in Bangladesh?

Specific intervention details varied in each A&T country, but, in Bangladesh specifically, the initiative included interpersonal communication (IYCF counseling with pregnant women and mothers of young children) and community mobilisation (local meetings with stakeholders and village theatre performances); these components were delivered by BRAC, a large non-governmental organisation (NGO) with a network of community-based volunteers, in areas assigned to A&T interventions. (Mass media (television commercials and radio stories about IYCF) and policy advocacy (dissemination and the creation of a National IYCF Alliance) were also conducted country-wide across intervention and comparison A&T areas, but this study focuses on the interpersonal communication (IYCF counseling) and community mobilisation components of A&T in Bangladesh.)

The researchers use data from a cluster-randomised controlled trial (RCT) design, including repeated cross-sectional surveys with health workers in 2010 (baseline, n = 290), 2014 (endline, n = 511), and 2017 (post-endline, n = 600). Health workers in 10 sub-districts were trained and incentivised to deliver intensified IYCF counseling and participated in social mobilisation activities, while health workers in 10 comparison sub-districts delivered standard counseling activities. Multivariable difference-in-difference linear regression models, including worker characteristic covariates and adjusted for clustering at the survey sampling level, are used to compare differences between groups (intervention vs. comparison areas) and over time (baseline, endline, post-endline).

At endline, health workers in intervention areas discussed significantly more IYCF topics than those in comparison areas (4.9 vs. 4.0 topics, p < 0.001), but levels decreased, and the post-endline gap was no longer significant (4.0 vs. 3.3 topics, p = 0.067). Comprehensive refresher trainings were protective against deterioration in service delivery. Between baseline and endline, the intervention increased health workers' knowledge (3.5-point increase in knowledge scores in intervention areas, vs. 1.5-point increase in comparison areas, p < 0.0001); this improvement persisted to post-endline, suggesting a sustained programme effect on knowledge. (However, while knowledge may be a necessary ingredient to achieving—and sustaining—behavior change, it may not be sufficient.) Job satisfaction and readiness both saw improvements among workers in intervention areas during the project period (baseline to endline) but regressed to a similar level as comparison areas by post-endline.

Comprehensive refresher trainings potentially protected against the deterioration of the primary outcome (service delivery). Workers in intervention areas who had less-comprehensive refresher trainings experienced the largest declines in IYCF counseling by post-endline. However, more comprehensive IYCF refresher training was not associated with any difference in knowledge (intermediate outcome) across treatment groups, between endline and post-endline. There was no apparent interaction between comprehensiveness of refresher training and job satisfaction or job readiness in either group over time. Furthermore, there was no apparent effect of incentives, whether presence of any incentive or the amount of incentive, on quality of IYCF service delivery nor on intermediate outcomes.

"If, as in this case, an intervention can have lasting effects on health worker knowledge, but cannot continue to make material contributions that may help translate this knowledge into action - for example, incentive payments or job readiness - what are reasonable sustainability expectations?" The researchers urge that contextual factors be considered when preparing for and assessing sustainability. For example, previous research has explored how social factors, like religious norms, may affect BRAC workers' job performance. There may be other factors, including macroeconomic factors or structural changes in the healthcare system, that may impact intermediate outcomes, like job satisfaction for health workers.

Programmes of limited duration may seek to assess the status of and invest in protective factors identified in this study (e.g., refresher trainings) to encourage sustained impact of improved service delivery. "More research that attempts to disentangle sustainment of effects across a program's theory of change may help policymakers prioritize areas for continued investment after programs end, in order to catalyze ongoing impacts."