In order to understand what was driving improvements in coverage, ARISE chose to conduct research in what ARISE called “positive change countries”, meaning those countries had had a positive increase in DTP3 coverage in the past 10 years (2000-2009). ARISE then focused within those countries on districts with recent positive change in DTP coverage. Ultimately, ARISE chose Ethiopia, Ghana, and Cameroon for study.
Districts within those countries were then selected if they had at least 65-70% coverage three years prior, and had reached over 80% DTP3 coverage in 2009-2010. Focusing on recent performance improvement experience allowed ARISE to reduce recall bias, and improve their chances of speaking with respondents who had been part of the performance improvement process. ARISE also chose, a priori, a “steady” district in each country that had similar baseline performance in 2007, but where coverage had not improved over the same period of time.
This study took a fundamentally different research approach than most immunization performance studies. ARISE purposely selected positive change districts so that they could study the following question: “What do you think has contributed to this rapid improvement in coverage?” Most analyses of immunization performance focus on the barriers to coverage improvement rather than the positive drivers of change. While challenges and barriers were not discounted, ARISE focused on their investigation and interviews on identifying the items that had worked well.
The experience of the positive change districts was then compared to the expereience of the “steady” district to see what was different. ARISE asked fundamental questions, such as:
- Were the items that worked well in the other districts present or absent in the steady district?
- If present, were they working as well?
- If they weren’t working as well, why?
ARISE asked these questions of a variety of respondents, to get a as close to a 360 degree perspective on immunization performance as possible. These interviews included national, regional/ zonal, district, sub-district, health center, health post, and community members. In all, ARISE spoke with about 300 individuals in 12 districts within the three case study countries.
In addition, ARISE conducted a rapid situation analysis of the Expanded Programme on Immunization (EPI) at the national and district level to determine whether there were any gaps or differences in the essential components of the EPI program, using a combination of available program data, record review, interviews, and observations.
However, coverage data proved to be problematic. In most situations, immunization performance could only be verified once the team had been in the district and examined the coverage data a very disaggregated level. While this did not invalidate the research approach used, it highlighted the importance of data quality and the difficulty relying on reported coverage data alone as a measure of RI system performance at the district level.