Integration of a Digital Health Intervention Into Immunization Clinic Workflows in Kenya: Qualitative, Realist Evaluation of Technology Usability

"Human-centered design and human-factors methods can assist during pilot stages to better align systems with users' needs and again after scale-up to ensure that interventions are suitable for all user settings."

In an effort to increase vaccination coverage in low-resource settings, technological tools have been introduced to better track un- or underimmunised children, communicate vaccination appointment reminders, improve data management practices, and provide improved access to immunizsation coverage data. However, digital health intervention (DHI) projects often fail to be adopted by users or to demonstrate their potential value, owing to the poor understanding of users' needs and the implementation context. This paper describes how healthcare workers (HCWs) in Kenya integrated an electronic immunisation registry (EIR) into their immunisation clinic workflows, using qualitative human-centred design (HCD) and implementation science research methodologies. The researchers use these findings to inform the development of a refined programme theory on the registry's usability.

The EIR was introduced to all immunising facilities in Siaya County starting in 2018. A tablet-based EIR application was designed and developed using open-source software (OpenSRP-OpenMRS) adapted for the Kenyan health care setting to reflect the country's immunisation programme schedule, closely reflecting the standard paper-based reporting forms used by HCWs during immunisation sessions but with clinical decision support features to guide clinicians. The EIR was rolled out to all 161 immunising facilities throughout the county through several phases of training. Across the 12 facilities included in the study, 19 HCWs were interviewed, and 58 workflow sessions were observed.

Generally, there was a high level of agreement (≥75%) with EIR usability; however, users disagreed on the EIR being well integrated into their workflow, with 32% (6/19) disagreeing on having a good workflow when completing dual data entry and 42% (8/19) disagreeing on having enough staff to adequately use the EIR in an immunisation clinic.

The researchers' initial programme theory illustrated EIR usability as a cyclical relationship among data demand, data capture and accessibility, and data used for decision-making, which covers activities such as identifying clients due for vaccination, scheduling future visits, and retrieving clients' contact information. For the refined programme theory, the researchers added the intermediary conditions of system maturity, user's role, and data usability based on empirical study findings, including their own. They carried through the inclusion of the enabling environment from the initial programme theory because they found that factors such as high user workloads, untrained staff members, or poor internet connectivity could greatly influence EIR usability.

Using HCD methods to understand the fit between users and technology, the researchers distilled findings into 4 rules that consider the importance of a user's workflow and the enabling environment:

1. Ensure that users complete training to build familiarity with the system, understand the value of the system and data, and know where to find support.
2. Confirm that the system captures all data needed for users to provide routine health care services and is easy to navigate.
3. Identify work-arounds for poor network, system performance, and too few staff or resources.
4. Make users aware of expected changes to their workflow, and how these changes might differ over time and by facility size or number of patients.

Thus, DHIs "should serve as mechanisms to improve health care provision and data use; however, they need to adapt to the realities of the users and their enabling environment to become viable data management tools."

In conclusion: "Implementers should consider workflows during the design and implementation phases and ensure they are evaluating how workflows may vary by site and user, to better align the system with reality. HCD and human-factors research can assist during a digital intervention's pilot stages in time to make system changes, in addition to being used after scale-up to ensure interventions are acceptable in all user settings.