The use of continuous household surveys to generate timely data for annual programme outcome monitoring : Experience from the Integrated Family Health Program in Ethiopia

Lack of timely outcome-level data for evidence-based decision-making presents important challenges to most health and development programmes in low-income countries (Hirschhorn et al. 2013; Maluka et al. 2010; Mutemwa 2006). The main reasons for this include the lack of time, resources and expertise to generate outcome-level data on a regular basis. Outcome-level data that reflect local variations are generally unavailable and prevent health and development professionals from monitoring interventions and adapting programmatic activities.


Introduction
Lack of timely outcome-level data for evidence-based decision-making presents important challenges to most health and development programmes in low-income countries (Hirschhorn et al. 2013;Maluka et al. 2010;Mutemwa 2006).The main reasons for this include the lack of time, resources and expertise to generate outcome-level data on a regular basis.Outcome-level data that reflect local variations are generally unavailable and prevent health and development professionals from monitoring interventions and adapting programmatic activities.
Variations of the continuous survey concept have already been applied in different settings.For example, those conducted by the US Bureau of Labour Statistics (2008) report on monthly economic indicators and are a key source of real-time information for decision makers.In Malawi, sub-district continuous surveys were implemented during a period of rapid scale-up of malaria control efforts in 2010 (Roca-Feltrer et al. 2012).The implementer has reported that these surveys were feasible and affordable, had provided valuable information about where to target malaria control efforts and were recommended as a promising district-level monitoring and evaluation tool for short-term processes.In Peru, a continuous survey was implemented nationally for five consecutive years (2003)(2004)(2005)(2006)(2007)(2008) and concluded to be successful in its design and implementation, and recommended for other countries that currently rely on large-scale surveys (USAID 2016).More recently, continuous household and health facility surveys have been implemented in Sengal (DHS 2013).
A number of existing large-scale community-based surveys have been conducted to provide accurate data on the health status of low-income countries at one point in time, such as Demographic and Health Survey (DHS) (Central Statistical Agency 2016), Multiple Indicator Cluster Survey (MICS) (UNICEF MICS 2016) and Health Information System (HIS) (Mutale et al. 2013) data.These surveys unfortunately have several limitations: Firstly, the periodicity of these surveys is usually once every 2-5 years, which means that programme managers have to wait several years to assess performance of interventions on key health outcomes of interest in order to consider actions for course correction (Central Statistical Agency 2016;UNICEF MICS 2016).HIS data which capture health service statistics at individual health facilities and aggregated up to the district, regional and national levels have the potential for supporting the programmatic decisionmaking.Unfortunately, HIS data often have limited practical utility because of problems with completeness, timeliness, representativeness, accuracy of data capture and aggregation at multiple levels of the health system.Moreover, HIS data often do not have household-level statistics, which limits the utility of this data source (Mutale et al. 2013).There is limited recorded continuous household survey implemented in Ethiopia.
Since 2011, the Integrated Family Health Program (IFHP) continuous household survey was proposed as a possible means of routinely collecting data on key health outcome indicators of interest.The survey was conceptualised to be undertaken in the regions, with a programmatic level estimate for a broad range of indicators, using sample sizes that have reasonable precision for regional and outcomelevel analysis.The IFHP is a project funded by USAID and implemented by Pathfinder International and John Snow, Inc. since July 2008.The project focuses on providing integrated family planning (FP); maternal, newborn and child health (CH) services; as well as improving the quality of reproductive health services specifically in public-sector facilities of the four major target intervention regions (Tigray, Amhara, Oromia and SNNP) and to a lesser extent in Benshangul Gumuz and Somali regions in the country.The programme supports and strengthens the primary health care services through health extension program (HEP) at community and health post (HP) level.The IFHP provides different types of technical and logistics support at all levels of the public system in 300 woredas (districts) of which 286 are found in the survey regions.For programme outcome monitoring, IFHP conducted a continuous household survey annually for 6 years (2011-2016) for the purpose of meaningful time series analysis of key health outcomes (IFHP 2016).
This manuscript describes the IFHP experience in implementing the continuous household surveys in project intervention areas as part of programme outcome monitoring.In our understanding, this is an example of implementing continuous household surveys designed to generate timely data and monitor programmatic health outcomes in Ethiopia.
The IFHP continuous survey represents a new paradigm that places equal weight on data systems for course correction as well as programme outcome monitoring.The aim of this report is to discuss the implementation of the continuous household surveys in the IFHP.

Survey implementation
The IFHP continuous survey is based on a multistage crosssectional survey design.IFHP had been supporting 286 woredas (districts) in its four major target regions (77 in Amhara, 120 in Oromia, 54 in SNNP and 35 in Tigray).Sixteen cluster-level offices (CLOs) were established in these regions (four in Amhara, six in Oromia, three in SNNP and three in Tigray) in order to support programme implementation in the intervention areas.All 16 CLOs were considered for the survey.A sampling frame was prepared by listing the Woreda Health Offices (WorHOs) under each CLO, health centres (HCs) under each WorHOs and HPs under each HC.A total of 128 woreda health offices, 256 health centres and 512 HPs were selected randomly using proportion to the size of the region.If the HP was randomly selected for the IFHP continuous household survey, then the Keble (village) associated with the selected HP was chosen for the next stage of sampling.From each selected Keble, five households (HH) were selected using a random-walk technique (Figure 1).As a result, a total of 2560 households were selected.If there is more than one eligible respondent found in a household, simple random sampling was used to select one eligible respondent.
A household questionnaire was developed in consultation with programme technical advisors.Key questions on CH, maternal and newborn health (MNH), FP and nutrition were considered.Regional and CLO staff members received training to use the survey questionnaire and were responsible for data collection in their respective catchment areas.Data collection for the IFHP continuous survey was conducted annually between January and March.

Consider all cluster offices
Eight Woreda health offices selected randomly from each cluster Two health centres selected randomly from each of the selected Woreda health offices Two health posts selected randomly from each of the selected health centres One community and five households selected randomly from each of the selected health post with different age categories = 2560 N FIGURE 1: Sampling process flow chart.
The data management process was managed by the IFHP Monitoring and Evaluation (M&E) team at regional and country office levels.Data checks were conducted by the region and the country office (CO) M&E team.Access database was used to enter the data and finally transferred to SPSS Version 20 for analysis.Ten per cent of the questionnaire was selected and re-entered and cross-checked for data consistency and completeness.It was observed that about 99% of the questionnaires were entered accurately.

Results
Six rounds of data collection for the IFHP continuous survey took place throughout the life of the project.Trend analyses were conducted with 95% confidence intervals around all point estimates.These results were shared with the regional programme manager and programme advisors during project review meetings.As a result, the programme made significant efforts to ensure the use of FP as a norm in the community -the FP-focused interventions have generated more than 23 million couple years of protection.For example, data suggest that 27.4% of women at baseline in 2008-2009 and 40.1% of women at first survey in 2011 aged between 15 and 49 years used a modern method of contraception.
Learning about these gaps mid-implementation meant that the project had introduced different initiatives to train health extension workers (HEWs), a cadre of front-line health workers and adapt HEWs refresher courses as necessary.Drawing on lessons, IFHP introduced different programme support strategies to increase FP service access and uptake, such as (1) extending clinical practicum session for HEWs from 2 days to 3 days to ensure sufficient time to exercise their skills; (2) increasing the number of practicum sites and clinical trainers from 3-4 to 6 to ensure a sufficient level of client demand and HEW competency; (3) increasing the amount of post-training commodities and consumable supplies for routine services from 30 to 60 client supplies per HEW, thus meeting client demand and allowing immediate initiation of services after the training; (4) providing gap-filling of commodities and consumable supplies support during follow-up visits to ensure uninterrupted continuation of services; and (5) strengthening the linkage between HCs and their catchment area HPs through back-up support to maintain competency of HEWs' skills and ensure the provisions of methods not available at the HP level.As a result, the trend of survey data reports that, women between the ages of 15 and 49 used a modern method of contraception increased and reached 55.6% in 2016 (Figure 2), while longacting contraceptive prevalence rate (LCPR) also increased from 4.8% to 14.7%.
Results in skilled maternity service uptake -including antenatal, delivery, and postnatal care -are encouraging.This is a major government priority area.In 2008, only 7% of births were attended by a skilled provider, and antenatal care and postpartum care were sparse and inconsistent.As shown in Figure 3, the survey data show women with children 0-11 months who had at least four or more antenatal care visits at a health facility increased from 30.8% in 2012 to 64.6% in 2016.Since that, more than 7.2 million women delivered with the assistance of skilled birth attendants.Close to 4 million women received at least one postnatal care visit.A similar trend is observed in the percentage of skilled attendance at deliveries that occurred in a health facility, showing an increase from 13.3% in 2012 to 66.4% in the 2016.With respect to child feeding practice indicators, there is great improvement during the project implementation period (Figure 4).Finally, several promising practices were identified and recognised nationally as well as our shared experience with the national and international audiences.The survey learning evidence was used to inform programme decisions and actions.For example; data from the survey revealed that certain health facilities and HPs lacked the necessary supplies for functional oral re-hydration therapy, while others lacked a designated location and board for information-sharing between facility staff and clients.Learning about these gaps, the project had the opportunity to provide and distribute oral re-hydration therapy supplies and information materials to health facilities and HPs to   improve their responses to dehydration and diarrhoea.In addition, the project provided pin boards to health facilities and HPs so that they were able to make information about health and child feeding practices information or post available to clients.

Discussion
The continuous survey approach involves the application of the continuous household survey methods with programmespecific context.The intent of this discussion is to contribute to a dialogue on strengthening health systems of developing countries with the state-of-the-art methods for monitoring, evaluation and timely data use.Every year, the data were synthesised into a report to support the evidence-based decision for monitoring and evaluating team and programme staff working at different levels.

Lessons learnt
Outcome-level data that reflect local variations are generally unavailable and they prevent health and development professionals from monitoring interventions and adapting programmatic activities.The following lessons were learnt through survey activities: • Continuous household survey and regular analysis of data is critical to identify challenges and bottlenecks so that timely adjustment of programme direction can be ensured and additional programme needs can be identified.• Integrating household survey is important for programme outcome monitoring without waiting for costly mid-term or end-term project evaluation.• It is essential to strengthen programme monitoring systems to cater to the organisation's data need annually.

Conclusion
In conclusion, continuous survey process, as part of programme activity, helped to produce outcome-level data for strategic decision-making on a timely basis, and it helps to increase the value for money.
The survey process has fostered improved information utilisation and ensured learning and mid-implementation corrections in programme implementation.Integrated Family Health Program reported that these surveys were feasible and affordable, and had provided valuable information about where to target programme interventions.Integrated Family Health Program recommends that other programme implementers should be plan integrated continuous programme outcome monitoring mechanism as part of programme design rather than waiting for mid-term or end-term project evaluation.Infants age 0-11 months that started first milk (colostrum) and exclusive breastfeeding(2011)(2012)(2013)(2014)(2015)(2016).