Development and Formative Evaluation of the e-Health Implementation Toolkit (e-HIT)

This paper reports on the development and formative evaluation of an e-Health Implementation Toolkit (e-HIT) which aims to summarise and synthesise new and existing research on implementation of e-health initiatives, and present it to senior managers in a user-friendly format. The toolkit was developed for the United Kingdom (UK) National Health Service (NHS). The paper is published in the October 2010 issue of the journal BMC Medical Informatics and Decision Making.

According to the authors, there is significant variability in the success of e-health implementations, and the evaluations of them are focused on "organisational issues related to implementation, with little attention paid to the impact of new technologies on workload, inter-professional relationships, and communication between health professionals and patients." This paper describes "the development and formative evaluation of an e-Health Implementation Toolkit (e-HIT) [Footnotes have been removed by the editor]. The e-HIT was developed explicitly to bridge the managerial:research interface, by making the results of a large body of research into e-Health implementation accessible to senior managers in a simple, user-friendly format....The aim of the e-HIT was to summarise and synthesise research evidence on factors that impede or facilitate implementation of e-Health initiatives and present this evidence in a format that could be easily digested and used by staff considering or planning an e-Health implementation."

Four constructs were used in a theoretical framework for evaluating data from a literature review on e-health initiatives:

• "Interactional workability (IW) refers to the degree to which the proposed technology enables (or impedes) interactions between health professionals and patients - e.g. a consultation.
  
• Relational integration (RI) refers to the way in which different professional groups relate to each other, and how well the proposed technology fits (integrates) with existing relationships, as well as the degree to which it promotes trust, accountability and responsibility in inter-group relationships.
  
• Skill set workability (SSW) refers to the degree to which the e-health initiative fits with existing working practices, skill sets, and perceived job role.
• Contextual integration (CI) refers to the degree to which the proposed e-health system fits (integrates) with the overall goals and structure of the organization (context), as well as the capacity of the organization to undertake the implementation...."

Based on the results, a toolkit was then developed for use in e-health project design and analysis. The information was developed as software by a web design company familiar with toolkit design. "....Additional material was written to inform users how the toolkit had been derived, its aim, how to download and complete it, and how to interpret the results." The toolkit included illustrative case studies. A formative evaluation was done on user feedback from senior clinicians, managers, and academics with experience of e-health implementations. "This feedback led to a number of changes, including redesign of the introductory section, allowing experienced users to bypass this, provision of explanatory mouseovers to assist with completion of individual questions, a more streamlined lay-out and a more detailed explanation of how to use the report section of the toolkit." A second round of evaluation yielded feedback resulting in minor changes. Page 6 of the paper shows a sample page screen shot of the toolkit. The e-HIT toolkit is available for downloading and unrestricted use.