Program Improvement Through Use of Process Evaluations

From Capable Partners (CAP) and FHI 360, this publication is part of a Capacity Building Series documenting the experiences of the CAP Botswana project in organisational development, and building the technical capacity of local civil society organisations in HIV Prevention, from 2008-2011. The purpose of the programme is to strengthen community-based responses to HIV prevention implemented by civil society organisations (CSOs), and help the organisations develop into strong and effective partners in the national HIV and AIDS response.

This publication describes the CAP project annual process evaluation methodology, outlines the implementation processes, details the major activities and tools used, and shares key results. The process, findings, tools, and results are described as relevant to other organisations involved in capacity building or implementing community-based programmes in Botswana and beyond.

"Process evaluations look at two critical areas of programme implementation - coverage and process. Coverage examines the end beneficiary of services by answering several questions including: What proportion of those who need services actually use them? Are the services reaching the intended target population as designed? and What are the demographics of those who are reached through services? Process looks at various aspects of implementation of services such as: How are clients identified and targeted for services? and What are the tools, processes used for managing the project implementation process for client satisfaction, delivery of quality and needed services and ensuring adequate coverage? The process evaluation was designed as a rapid two-day assessment to answer the above questions (Refer to Annex 1 for a Sample Agenda).

The main objective of the process evaluations was to:

• Assess how programmes were implemented, in order to improve their approach and procedures in the next year of implementation.
• Explore programme structure and content, the type of activities carried out, their relevance, frequency and manner of execution; perceptions of programme impact; relationships between organisation/programme and key stakeholders; and feedback from beneficiaries and others."

The methodology synopsis describes the various methods below (Refer to Annexes 3-8 for tools):

• Focus group discussions with:

1. One group of stakeholders
2. Three groups of beneficiaries
3. One group of facilitators (peer educators and field officers)

• Knowledge, Attitude and Practice (KAP) Surveys - administered among beneficiaries and implementers
• Behaviour Change Communication (BCC) observation - use of a BCC observation checklist to document and provide feedback on a delivered HIV prevention community outreach session
• Data Quality Audit - verification of data submitted in monthly reports

Findings and recommendations are charted on pages 10 and 11 in section 2.4.