About the Author(s)


Harriet R. Kagoya Email symbol
Management Sciences for Health, Windhoek, Namibia

Dan Kibuule symbol
Department of Pharmacy Practice and Policy, School of Pharmacy, University of Namibia, Namibia

Citation


Kagoya, H.R. & Kibuule, D., 2018, ‘Quality assurance of health management information system in Kayunga district, Uganda’, African Evaluation Journal 6(2), a238. https://doi.org/10.4102/aej.v6i2.238

Original Research

Quality assurance of health management information system in Kayunga district, Uganda

Harriet R. Kagoya, Dan Kibuule

Received: 22 Apr. 2017; Accepted: 03 Aug. 2018; Published: 12 Sept. 2018

Copyright: © 2018. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background: An efficient health management information system (HMIS) improves health care delivery and outcomes. However, in most rural settings in Uganda, paper-based HMIS are widely used to monitor public health care services. Moreover, there are limited capabilities and capacity for quality HMIS in remote settings such as Kayunga district.

Objectives: The quality assurance practices of HMIS in health centres (HCs) in Kayunga district were evaluated.

Method: A cross-sectional descriptive study design was used to assess the quality of HMIS at 21 HCs in Kayunga district. Data were collected through in-depth interviews of HMIS focal persons as well as document analysis of HMIS records and guidelines between 15 June 2010 and 15 July 2010. The main outcomes were quality assurance practices, the HMIS programmatic challenges and opportunities. The practice of HMIS was assessed against a scale for good quality assurance practices. Qualitative data were coded and thematically analysed, whereas quantitative data were analysed by descriptive statistics using SPSS v22 software.

Results: All the 21 HCs had manual paper-based HMIS. Less than 25% of HCs practised quality assurance measures during collection, compilation, analysis and dissemination of HMIS data. More than 50% of HCs were not practising any type of quality assurance during analysis and dissemination of data. The main challenges of the HMIS were the laborious and tedious manual system, the difficulty to archive and retrieve records, insufficient HMIS forms and difficulty in delivering hard copies of reports to relevant stakeholders influenced quality of data. Human resource challenges included understaffing where 43% of participating HCs did not have a designated HMIS staff.

Conclusion: The HMIS quality assurance practices in Kayunga were suboptimal. Training and support supervision of HMIS focal persons is required to strengthen quality assurance of HMIS. Implementation of electronic HMIS dashboards with data quality checks should be integrated alongside the manual system.

Introduction

Problem statement

Remote health care settings are at risk of inadequate health care service monitoring and delivery. In the absence of established quality assurance measures for health management information systems (HMIS), the data output often has undiminished errors and may not be valued enough to be used for the intended purpose or may be misleading if used. In addition, health decisions based on poor quality data hinder effective implementation of health improvement plans (Ramesh et al. 2012). Thus, the implementation of a quality assurance system for HMIS yields critical signals for actual and potential errors during data collection and processing, identifies the need for technical support or indicates the necessity to adapt the system to new demands from health programmes or decision-makers (Kasambara et al. 2017; Kyalo & Odhiambo-Otieno 2017; Muhindo & Joloba 2016; Nkolo et al. 2008; Qazi & Ali 2009; Upadhaya et al. 2016). Also, the success of HMIS depends on the integration of the system in the health sector’s processes as well as political and technical support by the main stakeholders. Unfortunately, several studies depict existing gaps and limited implementation of quality assured HMIS data, a major public health concern. Generally, in most remote health settings in low- and middle-income settings and countries (LMIC), such as Kayunga district in Uganda, there is limited information about practices of quality assurance for HMIS data as well as the performance of the HMIS that generates data. The World Health Organisation (WHO) developed a framework for strengthening health information systems (HISs) in such settings; it gives limited guidance on quality assurance practices for the data collection and utility (WHO 2010). These gaps necessitated this study to assess the data quality assurance practices, challenges and opportunities existing for data quality improvement.

Literature review

The health management information system and its importance

A health management information system is a critical tool for effective and efficient health care delivery. According to the WHO, the HIS is responsible for generating, compiling, analysing and synthesising, communicating and using data for decision-making, and ensuring their overall quality. Among other functions, the information system provides information that supports patient and health facility management (WHO 2008). District health information systems (DHISs) provide inputs for the formulation of national and regional policies. Health management information systems provides data that are needed by policy-makers, clinicians and health service users to improve and protect population health (Garg & Garg 2015). Accurate, relevant and up-to-date information is essential to health service managers if they are to recognise weaknesses in health service provision and take actions to improve service delivery. The HMIS is one of the components of HIS which are critical for achieving public health goals and strengthening health management (DFID Health Resource Centre 1987).

Health management requires monitoring health status of the population, provision of services as to the coverage and utility, drugs stocks and consumption patterns, equipment status and availability, finances and personnel on a regular basis. This requires timely and accurate information from various sources. Health information systems in developing countries like Uganda mainly focus on epidemiological outcomes, service utilisation and finance. They generate little of the socio-cultural data needed for developing and adjusting health services and disease control programme to local health-related perceptions, values and resources.

In Uganda, the HMIS was introduced in 1997 (Hotchkiss et al. 2010). The HMIS is a database system in which ‘raw data’ are stored and transformed into information (Lippeveld, Sauerborn & Bodart 2000). Quality assurance with respect to this study referred to the planned and systematic actions that were done or deemed necessary to provide adequate confidence that the HMIS structure and its components performed satisfactorily in health service.

Quality assurance practices for health management information systems

Measures to ensure the quality of routinely collected data such as that obtained through the HMIS include allocating staff to handle data, training of the staff to enhance their knowledge and create a sense of ownership of data, provision of needed resources such as files and reporting forms, quality support supervision visits to health workers, meetings which discuss findings rather than criticise health workers, and redesigning data collection forms and procedures (Mavimbe, Braa & Bjune 2005). For this study, quality assurance practices were grouped according to points of data management through the levels – at primary collection points, data compilation, analysis stage, reporting, storage and at dissemination point.

General challenges with health management information systems

There is a low demand for information because of lack of awareness by health policymakers and programme managers of the strategic importance and practical usefulness of health information for planning and management results (Bodavala). Where HMIS management is weak, data will not be used to plan, control or evaluate services rationally. Benefits of HMIS remain largely unrealised despite HMIS’s potential of being a strong health system tool (Upadhaya et al. 2016).

Health management information systems faced key challenges of poor data quality, inaccuracy and untimeliness (Selvaraju 2000). Health care providers both in the public and private sectors collect and submit data for decision-making. The data, however, are often not helpful for health management decision-making because they are incomplete, inaccurate, untimely, obsolete and unrelated to priority tasks and functions of local health personnel. Some types of errors or biases common in data collection which affect data quality include sampling bias, non-sampling error and subjective measurement (Selvaraju 2000; Upadhaya et al. 2016). Some data quality issues to consider include coverage, completeness, accuracy, frequency, reporting schedule, accessibility and power (Fraser et al. 2005; Selvaraju 2000).

Systems should collect only data that are needed for use. However, the information requested and collected from the primary health care and other hospitals every month is exhaustive and not warranted. Excessive information collection only burdens the system and generates into carelessness and ‘somehow to fill it and forget’ concept. The HMIS has been characterised by the absence of feedback; this defeats the purpose of information collection.

With the influx of donor funds to Uganda, some of which are tied to performance, there is a growing need for quality data to support decision-making (WHO 2008). Increase in international funding for health had been accompanied by accelerated demand for more and better statistics (Boerma & Stansfield 2007). Although timely HMIS data are required to support management decisions, HMIS has been found to be of poor quality (Hotchkiss et al. 2010).

Challenges faced in ensuring good quality of health management information system data

There are a number of constraints to implementing comprehensive HMIS with data quality assurance measures. These include lack of access to information and proper feedback by the Ministry of Health (MoH) to hospitals and lower level units; delayed feedback on the information provided by the different health institutions; lack of technical as well as managerial human capacity needed to implement a proper HMIS (Kyalo & Odhiambo-Otieno 2017; Qazi & Ali 2009). The link between HMIS and continuous medical education (CME) is non-existent because of an insufficiently recognised role of HMIS for CME in the health sector (Fenenga & de Jager 2007).

Inadequate infrastructure, human resource and technical expertise, especially in LMIC, negatively impacts on data quality. In addition, there is a lack of effective and comprehensive systems to gather the needed health data (Eldis Health Key Issues: HMIS Uganda). Another challenge with HMIS are the behavioural and motivational factors of the people collecting and analysing the data where health care providers equate information systems with filling endless registers, collating and compiling health data, conducting minimum data analysis, and sending out reports without receiving adequate feedback (Leon, Schneider & Daviaud 2012). Other challenges include getting the coding right, human resource capacity, data storage and use; ensuring that the people filling in the forms at clinic level are skilled enough to report accurately; and ensuring that people focus on the technical aspects of data storage and analysis, rather than on making sure the data are correct (Grant et al. 2015). The paper-based report system has made some districts in Uganda send reports with glaring gaps for many years.

The challenges are similar to those identified in other studies. A cross-country situation analysis of information systems for mental health in six LMIC found substantial policy, human resource and inadequate infrastructure, few HMIS experts, and inadequate technical support and supervision to junior staff for mental health HMIS (Upadhaya et al. 2016). The study recommended that greater technical support and resources be input to strengthen routine HMIS. A study conducted in Mozambique showed that data quality assessment (DQA) and validation were not prioritised, for example, in meetings (Mavimbe et al. 2005). Several studies have reported inconsistencies in data reporting as well as poor support mechanisms to ensure data quality at the district level. A study in Kenya found inconsistent reporting ranging from a minimum of 6% in December 1997 to a maximum of 44% in February 1996 (Gething et al. 2006).

Opportunities for improving quality of health management information system data

Quality of data may be assured or improved through provision of standardised reporting forms and manuals; training and quality support supervision of health facility staff who collect, analyse, report and disseminate data; meetings that discuss and validate the data presented in reports; and providing adequate feedback mechanism to the producers of data at the remote sites (Mavimbe et al. 2005). Feedback may be provided during dissemination in quarterly review meetings at the MoH, district and health centre (HC) level in Uganda (Upadhaya et al. 2016). Training health workers on the basic skills to monitor their own work could enhance ownership of the generated information and ultimately improve quality, as noted from experiences in Kyrgyzstan and in South Africa (Weeks et al. 2000; Williamson, Stoops & Heywood 2001)

Ensuring effectiveness and quality of health management information systems

In developing and implementing the HMIS, there is a need for strong political backing, developing a culture that values and uses information, involving all levels in changes to HMIS, starting with improving the paper-based system and ensuring that the feedback loop is continuous and reliable. There ought to be a long-term commitment to improve training and career structures for HMIS personnel working in the health and other social sectors to improve data systems. Also, improving technology and methods of measurement; improving capacity to collect and analyse data; establishing norms and standards for health measurements; and reporting valid and comparable assessments of inputs, service delivery and achievements are important aspects of monitoring and data quality assurance. New technologies can ease data collection, compilation and exchange, but clear data quality standards are still needed to ensure optimal value of HMIS data (WHO 2008).

With all the good practices, challenges and opportunities described in the preceding sections, it was not clear what practices of quality assurance of HMIS existed in Kayunga district, and to what extent any existing opportunities for data quality assurance had been exploited. It was therefore imperative that this study should be conducted to assess existing data quality assurance practices, challenges limiting implementation and ways of exploiting opportunities for data quality improvement in the district.

Key focus

Evaluating quality assurance practices, challenges and opportunities for HMIS in Kayunga district.

Objectives

The study assessed the extent of the data quality assurance practices as well as challenges and opportunities in implementing HMIS in Kayunga district.

Contribution to field

The findings from the study provide information on existing data quality assurance practices, challenges and opportunities for improvement of HMIS data quality in Kayunga district, and health care services in similar facilities, and LMIC.

A user-friendly HMIS is at the core of any successful public health system and will assist hospital managers to align health system resources with the needs of service users, link performance measurements to accountability, monitor health-related activities to help assess what works and what doesn’t, and contribute to organisational development.

Research method and design

Setting

This study was conducted in Kayunga district among 21 public and non-government HCs. Kayunga district is located in central Uganda. It consists of two counties and three health sub-districts. There were 23 HCs (19 government and 4 private-not-for-profit) and 27 clinics including maternity centres that were not categorised under specific levels at the time of this study. Convenience sampling was performed to select Kayunga district at the time of this study. It had a good representation of HCs by level and type, located in central Uganda within close proximity to the researcher’s base to enable cost-effective data collection.

Design

The study adopted a cross-sectional descriptive design and document analysis design. An analysis of HMIS documents and in-depth interviews of HMIS focal persons were conducted at 21 HCs in Kayunga district. Qualitative data were collected to document the quality assurance practices, challenges and opportunities for effective implementation of quality assurance practices in the district. The design explored practical solutions towards improving HMIS quality assurance practices in Kayunga district. The main study outcomes were quality assurance practices, challenges and opportunities for improvement. Qualitative data was coded and thematically analysed, whereas quantitative data were analysed using SPSS v22 software.

Procedure

Data were collected through in-depth interviews using structured interview guides with questions that had been pretested for validity and reliability prior to actual data collection. Data were collected from 15 June 2010 to 15 July 2010. Primary data were obtained by interviewing health facility staff in charge of compiling HMIS data for their respective facilities. Other vital information about quality assurance practices for HMIS data at the various facilities and levels was obtained by interviewing the health facility incharge who signed off the report (HMIS 105) that the HC submitted to the district every month and the district HMIS focal person who compiled the district’s HMIS 123 report that was submitted to the MoH.

Secondary data and additional information were gathered through studying publications which included books, published and unpublished papers, Internet material, official and unofficial documents and reports.

Background characteristics of the respondents decoded included age, sex, designation, number of years of service in the position and the highest level of education. Background characteristics of the health facilities decoded included level of the facility, geographic location and type of ownership of the HC.

Analyses

Data were entered in Epidata v3.1 for management. The unit of analysis was the HCs. Quantitative data were analysed by descriptive statistics using SPSS v22 software. Qualitative data were recorded in form of hand-written notes, coded, captured electronically and were manually analysed using thematic analysis to generate the themes and subthemes regarding the challenges and opportunities for implementation of HMIS.

Ethical considerations

This study was approved by the Makerere University School of Public Health as a field study. Permission to conduct the study in Kayunga district was obtained from the District Health Officer (DHO). Information from the HCs that participated in the study was recorded in such a way that it minimised disclosing the identity of the respective HC. The information gathered was kept confidential. The HC staff who participated as respondents gave informed consent following information given on the purpose, procedure, benefits and assurance of confidentiality of the information.

Results

Response rate and HC representation was 91.3% (21/23) of the targeted HCs, with one respondent per HC. The non-response from two facilities was because of unavailability of the focal person concerned with HMIS issues at the time of the study. Majority were HC level II (52%) and public facilities (81%) (Table 1). Twenty-one respondents (10 male and 11 female), that is, one from each HC participated in the study. Various cadres of staff were represented including 1 medical officer, 8 clinical officers and 7 enrolled nurses, among other cadres. The respondents were of ages ranging from 25 to over 48 years, had been in positions as incharge or records staffs for periods of time ranging mainly between 3 and 10 years, and were of varying levels of academic qualification including master’s and bachelor’s degree, diploma and certificate.

TABLE 1: Characteristics of health centres that participated in the study (n = 21).

All 21 (100%) HCs used a manual paper-based HMIS. Less than a third (29%) of respondents understood quality assurance. The researcher sensitised respondents on the meaning of quality assurance to ensure accurate response to interview questions. A quarter (25%) of HCs practised at least one quality assurance measure during collection, compilation, analysis and dissemination of HMIS data. More than 50% of HCs did not practise any type of quality assurance during analysis and dissemination of data. Health centres and health management information system-related challenges influenced the quality of data. Specific challenges reported by respondents included manual HMIS system that was laborious and tedious, difficult to handle and retrieve records, insufficient availability of the required HMIS forms and difficulty delivering hard copies of reports to relevant stakeholders. Human resource challenges included understaffing where 43% of participating HC had no HMIS staff allocated.

Quality assurance practices for health management information system data in Kayunga district

The practices were categorised according to stages of data handling, that is, practices at primary data collection points, compilation, analysis, reporting and storage, and during dissemination.

Quality assurance practices during data collection

All the 21 respondents practised at least one measure to ensure the quality of data that they collected (Figure 1). Twelve (57%) of the respondents practised at least two quality assurance measures.

FIGURE 1: Percentage of respondents who reported practising the stated quality assurance measure during data collection.

Quality assurance practices during health management information system data compilation

Less than 25% of the HCs in Kayunga had incharges check data, and 14% of the facilities reported not practising any quality assurance measure during compilation of data (Figure 2).

FIGURE 2: Quality assurance practices during data compilation.

Quality assurance practice during data analysis

More than half (52%) of the HCs in Kayunga were not practising any form of quality assurance during data analysis (Figure 3).

FIGURE 3: Quality assurance practices during data analysis. No single quality assurance practice mentioned for 11 (52.4%) of health centres.

Reported quality assurance practices during data reporting

Less than 25% of the HCs had their reports reviewed by the HC incharge for quality assurance before submission to the district (Figure 4).

FIGURE 4: Quality assurance measures practised during report compilation.

Reported quality assurance practices at data storage points

The majority (57%) of the HCs keep the data in a lockable store. However, less than 24% of the HCs practiced any one of the recommended quality assurance measures during storage of data (Figure 5).

FIGURE 5: Quality assurance practices implemented during data storage.

Reported quality assurance practices during data dissemination

The majority (52%) of the HCs in Kayunga were not practising any quality assurance measures before disseminating HMIS data. Less than 30% of the HC practised any one of the recommended quality assurance measures during dissemination (Figure 6).

FIGURE 6: Reported quality assurance practices during data dissemination. No single practice mentioned for 11 (52.4%) health centres.

Challenges faced in ensuring good quality of health management information system data

All 21 respondents enumerated challenges that they faced in ensuring good quality of HMIS data at their HCs of operation. All the 21 respondents mentioned at least one challenge, 14 respondents mentioned at least three challenges, while 8 respondents mentioned as many as four challenges. Thus, a sum total of 64 codes for challenges were enlisted from the 21 respondents. The challenges were thematically analysed and grouped into system-related, facility-related or human resource-related challenges (Figure 7).

FIGURE 7: Nature of mentioned challenges and corresponding percentage of occurrence.

Most (47%) of the challenges were system related. These included the manual HMIS that was laborious and tedious, difficult to handle and retrieve records, insufficient availability of the required HMIS forms and difficulty delivering hard copies of reports to relevant stakeholders. Some facilities mentioned improvising for HMIS forms using counter books which were ruled, but these had many inaccuracies. Also, the focal persons identified that HMIS forms did not provide options to disaggregate data for patient characteristics, for instance, there was no provision to enter the child’s gender for vitamin A and immunisation data.

Human resource (HR) challenges included understaffing where 43% of participating HCs did not have records staff; 52% of the HCs had only one records staff and only the hospital had two records staff. Only 2 of the 11 HCIIs (i.e. health centres at a parish manned by a senior nurse) had a records person, and the 2 were nongovernmental organisation (NGO)-owned facilities. As a result of understaffing, respondents stated that quality of data was usually compromised, either records were incomplete or the tallying was not performed daily as required leading to accumulation of unsummarised data. The staff handling the records, especially the HC incharges, felt incompetent to handle HMIS records because of no or limited training in records management. Also related to HR challenge was the fact that some of the staff had poor unreadable handwriting, while some did not fill all records because of laziness.

Facility-related challenges included poor handling of records where records were sometimes misplaced; lack of staff cooperation, especially when data clarification was needed; and change of health workers without proper handover. The attitude-related challenge was that staffs felt burdened by data responsibility and viewed this task as belonging to the incharge of the HCs.

Opportunities for improving quality of health management information system data

The most frequently mentioned opportunities were the continuous education (28%) in form of trainings and sensitisation either during seminars or on-the-job. Other opportunities included availability of staff some of whom were computer literate and support supervision (Figure 8).

FIGURE 8: Opportunities for improving quality of health management information system in Kayunga district.

On whether the mentioned opportunities had been exploited, 76% of the respondents stated that the opportunities had been exploited to some extent, such as through the advance purchase of registers from the National Medical Store or early purchase and ruling of counter books for use in data capture; district organised trainings and seminars with support from some NGOs like Protecting Families Against HIV/AIDS (PREFA); district review meetings that discussed anomalies, trends and progress; and support supervision of lower HCs.

The 24% respondents who stated that opportunities had not been exploited mentioned inadequate funding; staffing norms where HCIIs did not have records staff by standard; lack of electronic databases at the HCs and training of only the records staff rather than all staff in the HC who handled data.

Potential benefits and hazards

The study did not pose any serious risk to the respondents. Benefits of the study included insights for HCs on measures that they could use for improving and assuring quality of the information generated and disseminated through HMIS.

Trustworthiness

The findings and recommendations were based on data collected on HMIS quality assurance practices in Kayunga district at the time of the study in 2010. Since the time of the study in 2010 and the time of this publication, quality assurance practices may have changed. Nonetheless, the findings on quality assurance practices, challenges and opportunities provide insights for Kayunga district and the MoH of Uganda to improve the systems on which MoH relies for data that are used for decision-making.

Validity

The questions in the interview guide that was used for data collection were validated through pretesting on selected respondents. Thus, the findings are a close reflection of the quality assurance practices, challenges and opportunities of HMIS in Kayunga district at the time of the study.

Discussion

Quality assurance practices for health management information system data in Kayunga district

The study aimed to assess the quality assurance practices of HMIS data in Kayunga district in Uganda. The quality assurance practices during data collection, compilation, analysis and reporting were suboptimal and required strengthening to improve quality of health care delivery and outcomes.

Firstly, we found that the quality assurance practices during collection of HMIS data, that is, use of pre-printed registers, daily or regular review of data, ensuring completeness and accuracy of data at all stages, sensitisation of staff involved in data handling, supervision of records especially by the HC incharge varied widely across HCs and were not adhered to. In addition, there was limited and inconsistent availability of data collection or reporting forms at some HCs. This impacts on quality of data collected or reported and public health implications. A similar study in Rwanda also found variability in the availability of client or patient cards in the facilities in the study districts. The same study found, however, that the majority of the health facilities in the study districts accurately transmitted data from registers to health facility monthly reports (Innocent et al. 2016), at least ensuring quality of data during transmission from source document to aggregation report. Thus, there is a need to empower the HCs to generate forms for collecting and reporting HMIS data as well as adopt electronic tools for compiling patient and population level data in order to enhance efficiency.

Secondly, the quality assurance practices during data compilation also need strengthening, that is, the HC incharge checking the data, double-checks such as on the summations and aggregated data by staff responsible, immediate questioning and following up of any gaps identified, involvement of two staff, ensuring completeness and accuracy of each subset of data, regular meetings to discuss data and regular monitoring for easy detection of discrepancies. Ensuring completeness of data is a key component of quality assurance for which HCs practising it in Kayunga district are commended. This study however found limited validation or auditing of HMIS data by the HC incharge, but better than a study conducted in Pakistan which found that none of the reports reaching the District Health Office was checked for completeness (Mehmood et al. 2011). Such omission of quality assurance practices impacts on data quality aspects like completeness of facility reporting which subsequently affects completeness and accuracy of district reporting with as many as 33% of districts with monthly facility reporting rates below 80% in Uganda in 2011 (WHO 2011). Although a study conducted in Haryana state in India found average completeness of HMIS information for maternal and child health (MCH) services quite high at 88.5%, it also found over- or under-reporting of MCH services (Sharma et al. 2016), implying a shortfall in validity of the HMIS data. Thus, this calls for a system to validate the data that have been collected to prevent errors of omission, duplication or transcription. We suggest that routine audits of data, that is, every week, should be conducted to audit data collected as well as train staff involved in data collection on the importance of minimising errors. In addition, there is a need for electronic systems that detect and flag wrong data entry and reports.

Thirdly, during analysis, the quality practices were minimal, that is, there were limited audits by HC incharge to check and review current data, compare past and present data to detect anomalies, ensuring proper data from collection points through emphasis on keenness as well as involvement of two staff. It is recommended that routine comparisons of data are mandatory best practices in data quality management (HUD 2005). Thus, there is a need for a total quality management system for HMIS data that includes standard quality assurance practices and guidelines implemented at HCs.

Lastly, we also found suboptimal quality assurance practices during the HMIS report preparation. In most HCs, data were rarely double-checked; neither did the HC incharge review the report prior to dissemination, or data disaggregated and neither did the report writing involve two people to ensure high-quality data. Double-checking is a form of data quality assurance. Countries should develop manuals or tools to help the data quality checks such as the DQA procedure manual that Rwanda developed for routine DQAs at health facility level (Republic of Rwanda, Ministry of Health Rwanda 2016). Data quality assessments help to confirm the accuracy, completeness, consistency and timeliness of data.

To maintain quality of data during storage, safe custody was ensured by keeping files in lockable stores or cabinets with restricted access to staff. During dissemination, staff ensured quality to avoid corruption of data or information, through reviews and discussion of the report and further checks on data. Findings from other studies in this field also revealed practices such as the use of pre-printed forms and staff allocated to data collection.

Challenges faced in ensuring good quality of health management information system data

The manual HMIS was tedious and subject to many errors; HMIS forms were insufficiently available; and understaffing resulted in compromise of data as staff were tired because of work overload. Staff responsible for HMIS felt incompetent because of no or limited training in records management. Staff attitude was a challenge where HC staff felt overburdened by the record requirements, a task they viewed as the responsibility of the HC incharge. These challenges are similar to what Eldis Health Key Issues: HMIS Uganda mentioned including the fact that there was lack of an effective and comprehensive system to gather the needed health data; lack of technical and human capacity to implement a proper HMIS; non-existent link between HMIS and CME; and motivational factors of the people handling data, poor data storage and use. These challenges are similar to those reported by other studies regarding poor data quality, accuracy and timeliness (Selvaraju 2000); inadequate infrastructure; few HMIS experts; and inadequate technical support and supervision to junior staff (Upadhaya et al. 2016). A study in Pakistan also found limited (48%) human resources for HMIS function (Mehmood et al. 2011).

Opportunities for improving quality of health management information system data

Opportunities for improving quality of HMIS data included the provision of pre-printed HMIS forms; continuous education; sensitisation and training that the district organised; staff availability at all facilities, especially HCII that lacked records staff; availability of solar power which could be tapped for computer use; and regular support supervision undertaken by the district to the HCs on a quarterly basis. However, the opportunities had not been fully exploited as reported by 24% of the respondents.

Provision of HMIS manuals and pre-printed forms that served as a guide for record taking and compilation is critical for quality of data generated from such a system. The MoH HMIS manual highlighted that to ensure accuracy, written manuals describing the collection, compilation and use of the data and a training programme were to be provided. Training was also mentioned in the HMIS manual and that included both formal sessions and on-the-job follow-up. Supervision was equally critical for following up and ensuring that quality data were generated. Follow-up visits to the HC were emphasised in the HMIS manual and they had to be thorough and effective (MoH HMIS manual 2010). The WHO International Journal of Public Health (2005) mentions the need to improve training and career structures for HMIS personnel to make improvements to data systems. Murray, Lopez and Wibulpolprasert (2004) mention that improving technology and capacity to collect and analyse data are important aspects of data quality assurance.

Mavimbe et al. (2005) reports similar opportunities for quality assurance of data, including provision of standardised reporting forms and manuals; training and quality support supervision of health facility staff who collect, analyse, report and disseminate data; meetings that discuss and validate the data presented in reports; and providing adequate feedback mechanism to the producers of data at the remote sites. Timely feedback can improve data quality, and delayed feedback on data aspects such as data entry is a recipe for poor data (HUD 2005). In addition, training health workers on the basic skills to monitor their own work can improve data quality (Weeks et al. 2000; Williamson et al. 2001). The US Department of Housing and Urban Development Office of Community Planning and Development also recommended consistent and continual training of staff involved in data collection and entry for enhancing HMIS data quality (HUD 2005).

A study in Pakistan found that health facilities were not receiving timely feedback because data were not analysed at the higher level, and this limited the improvement by the reporting facilities, with a potential consequence of poor delivery of health services (Mehmood et al. 2011).

Practical implications

The findings on quality assurance practices for HMIS in Kayunga district, challenges and opportunities identified provide key insights for system improvement to ensure that the data generated through the HMIS are of the highest quality possible and thus useful for decision-making.

Limitations of the study

The interpretation of the findings should be contextualised by the fact that changes may have occurred in the implementation of HMIS at HCs in Kayunga district since 2010. Nonetheless, the results provide a benchmark for HCs that use paper-based HMIS systems in remote settings in Uganda and other LMIC. Although HC staff in charge of HMIS were targeted as the primary respondents for the study, not all HCs had staff specifically designated to collect and compile HMIS data. Other staff responded in the interviews; this may have limited details available on all the quality assurance practices implemented at the HCs in the district. Related studies should collect additional data through observation of practices for a more extended time than was done in this study. Despite the mentioned quality assurance practices, the researcher did not find out the extent of data quality from the HCs; this is recommended for further research. This study was conducted in one district in Uganda because of resource constraints. Future research may be conducted in multiple districts representing all regions of Uganda for generalisation of results and recommendations.

Conclusion and recommendations

The study revealed existence of quality assurance practices for HMIS in HCs in Kayunga, but the practices were suboptimal. The practices were quite limited as noted from the small number of HCs where the practices were implemented. The mentioned practices should be shared among HCs, implemented and replicated to a greater extent than was done at the time of this study.

Health centres faced challenges in ensuring good quality of HMIS data, ranging from system challenges to facility- and human resource-related challenges. Unless these challenges are minimised or eliminated where possible, quality of HMIS data will continually be affected.

A number of opportunities existed for improving quality of HMIS data in Kayunga district, some of which had been exploited by the district. The district should explore ways and find resources, both technically and financially to fully exploit the existing opportunities.

Training and support supervision of the HMIS focal persons are required to strengthen quality assurance of HMIS data. Implementation of electronic HMIS dashboards with data quality checks should be integrated, or it should replace the manual system to ease the laborious manual data entry and management, improve storage and ease retrieval of records at HC level.

The district should facilitate CME and training of HC staff responsible for handling data at various departments. As all staff in a way handled data, the district should facilitate training of all the staff, at least in the basic knowledge of the HMIS forms that they fill. This will ensure correct recording of data, minimise gaps and enhance completeness and accuracy of information. Health centre managers should ensure proper handover or transition from HMIS trained focal persons when staff transition from their roles.

District- and the higher-level support supervision to lower HCs should be enhanced for quality checks and improvement because these visits provide opportunities for on-the-job training. The district should consider this a priority and allocate resources.

Provision of HMIS forms in sufficient quantities is critical for enhancing quality data collection through the guiding information in pre-printed forms. The district should prioritise allocation of resources to ensure continuous availability of these forms.

Kayunga district management should encourage and facilitate regular reviews and meetings to discuss data, for example, emphasising the requirement of monthly HC staff meetings. This will provide opportunity for continuous sensitisation, iron out misunderstandings, sensitise staff to change attitude towards data and improve cooperation as staff will feel collective responsibility for their HC data. Also, meetings will enable information sharing and facilitate replication of quality assurance practices in the HCs.

Acknowledgements

The authors thank Dr Musisi Edrisa, District Health Officer of Kayunga district, for providing permission to conduct the study; respondents for their time; and HC staff for their useful insights on quality assurance practices in Kayunga district.

Competing interests

The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.

Authors’ contributions

H.R.K. conceptualised, designed and conducted the research, entered and analysed the data and wrote the manuscript. D.K. performed additional data analysis and reviewed the manuscript for publication.

References

Bodavala, R., 2012, Evaluation of Health Management Information System in India. Need for computerized databases in HMIS, viewed 22 April 2017, from https://cdn1.sph.harvard.edu/wp-content/uploads/sites/114/2012/10/rp176.pdf

Boerma, J.T. & Stansfield, S.K., 2007, ‘Health statistics now: Are we making the right investments?’, Lancet 369(9563), 779–786. https://doi.org/10.1016/S0140-6736(07)60364-X

DFID Health Resource Centre, 1987, Eldis health key issues: Health management information systems, viewed n.d., from https://www.scribd.com/document/57741460/HMIS

Fenenga, C. & de Jager, A., 2007, ‘Cordaid-IICD health programme Uganda: Health management information systems as a tool for organisational development’, The Electronic Journal of Information Systems in Developing Countries 31(1), 1–14. https://doi.org/10.1002/j.1681-4835.2007.tb00212.x

Fraser, H., Biondich, P., Moodley, D., Choi, S., Mamlin, B. & Szolovits, P., 2005, ‘Implementing electronic medical record systems in developing countries’, Journal of Innovation in Health Informatics 13(2), 83–95. https://doi.org/10.1002/j.1681-4835.2007.tb00212.x

Garg, R. & Garg, A., 2015, ‘District Health Information System (DHIS2) Software in India’, Advances in Computer Science and Information Technology (ACSIT) 2(10), 39–42.

Gething, P.W., Noor, A.M., Gikandi, P.W., Ogara, E.A., Hay, S.I., Nixon, M.S. et al., 2006, ‘Improving imperfect data from health management information systems in Africa using space–time geostatistics’, PLoS Medicine 3(6), 825–831. https://doi.org/10.1371/journal.pmed.0030271

Grant, T., Jenkins, H.E., Cobelens, F., Abubakar, I., Khan, A.J. et al., 2015, ‘Data for action: Collection and use of local data to end tuberculosis’, The Lancet 386(10010), 2324–2333. https://doi.org/10.1016/S0140-6736(15)00321-9

Hotchkiss, D.R., Aqil, A., Lippeveld, T. & Mukooyo, E., 2010, ‘Evaluation of the performance of routine information system management (PRISM) framework: Evidence from Uganda’, BMC Health Services Research 10, 188. https://doi.org/10.1186/1472-6963-10-188

Innocent, K., Onzima, R.A.D.D.M., Katongole, S.-P., Govule, P., 2016. ‘Quality and use of routine healthcare data in selected districts of Eastern Province of Rwanda’, International Journal of Public Health Research 4(2), 5–13.

Kasambara, A., Kumwenda, S., Kalulu, K., Lungu, K., Beattie, T., Masangwi, S. et al., 2017, ‘Assessment of implementation of the health management information system at the district level in southern Malawi’, Malawi Medical Journal 29(3), 240–246.

Kyalo, C.K. & Odhiambo-Otieno, G.W., 2017, ‘Towards a well-functional computerized health management information system: A case of Mbagathi County Hospital, Kenya’, International Journal of Computer Applications 161(3), 975–8887.

Leon, N., Schneider, H. & Daviaud, E., 2012, ‘Applying a framework for assessing the health system challenges to scaling up mHealth in South Africa’, BMC Medical Informatics and Decision Making 12(1), 123. https://doi.org/10.1186/1472-6947-12-123

Lippeveld, T., Sauerborn, R., Bodart, C. (eds.), 2000, Design and implementation of health information systems, World Health Organization, Geneva.

Mavimbe, J.C., Braa, J. & Bjune, G., 2005, ‘Assessing immunization data quality from routine reports in Mozambique’, BMC Public Health 5, 108. https://doi.org/10.1186/1471-2458-5-108

Mehmood, T., Ashraf, T., Mashhadi, S.F. & Roshan, R., 2011, Health management information system in district health care, viewed 22 April 2017, from http://www.pafmj.org/printdetail.php?id=41&t=f

Muhindo, R. & Joloba, E.N., 2016, ‘Health management information system (HMIS): Whose data is it Anyway? Contextual challenges’, Review of Public Administration and Management 4(2). https://doi.org/10.4172/2315-7844.1000190

Murray, C.J.L., Lopez, A.D. & Wibulpolprasert, S., 2004, ‘Monitoring global health: Time for new solutions’, British Medical Journal 329(7474), 1096–1100.

Nkolo, A., Nkwake, A., Mpeka, B., Adatu, F. & Kironde, S., 2008, ‘Aligning program information needs to existing health information management systems: TB/HIV collaboration in Uganda’, in XVII International AIDS Conference, Mexico City, Mexico, August 03–08, 2008.

Qazi, M.S. & Ali, M., 2009, ‘Pakistan’s health management information system: Health managers’ perspectives’, Journal of the Pakistan Medical Association 59(1), 10–14.

Ramesh, K., Shaikh, B.T., Chandio, A.K. & Jamil, A., 2012, ‘Role of health management information system in disease reporting at a rural district of Sindh’, Pakistan Journal of Public Health 2(2), 10–12, viewed 01 August 2018, from http://www.hsa.edu.pk/journal/pastissue.php

Republic of Rwanda, Ministry of Health, 2016, Data quality assessment procedures manual, viewed 22 April 2017, from http://www.moh.gov.rw/fileadmin/templates/HMIS_Docs/Hmis_SOPs/Rwanda_Procedures_Manual_for_DQA_Version_2016.pdf

Republic of Uganda, Ministry of Health, 2010, ‘The health management information system’, Health Unit Procedure Manual, viewed 22 April 2017, from http://www.health.go.ug/hmis/public/tools/HC_Revised_HMIS_Manual_Revised2_updated_.pdf

Selvaraju, S., 2000, Health information management: Malaysian experience, viewed 19 April 2017, from http://www.apami.org/apami2006/papers/Health%20Information%20Management%20Malaysian%20Experience.pdf

Sharma, A., Rana, S.K., Prinja, S. & Kumar, R., 2016, ‘Quality of health management information system for maternal & child health care in Haryana state, India’, PLoS One 11(2), e0148449. https://doi.org/10.1371/journal.pone.0148449

Upadhaya, N., Jordans, M.J.D., Abdulmalik, J., Ahuja, S., Alem, A., Hanlon, C. et al., 2016, ‘Information systems for mental health in six low and middle income countries: Cross country situation analysis’, International Journal of Mental Health Systems 10, 60. https://doi.org/10.1186/s13033-016-0094-2

US Department of Housing and Urban Development Office of Community Planning and Development (HUD), 2005, Enhancing HMIS data quality, viewed 20 April 2017, from https://www.hudexchange.info/resources/documents/EnhancingHMISDataQuality.pdf

Weeks, R.M., Svetlana, F., Noorgoul, S. & Valentina, G., 2000, ‘Improving the monitoring of immunization services in Kyrgyzstan’, Health Policy Plan 15, 279–286. https://doi.10.1093/heapol/15.3.279

WHO, 2008, Toolkit on monitoring health systems strengthening, Health Information Systems, viewed 18 April 2017, from https://docplayer.net/9176924-Toolkit-on-monitoring-health-systems-strengthening-health-information-systems.html

WHO, 2010, Health information systems, 2nd edn., WHO, p. 72, Hershey, PA: IGI Global. https://doi.org/10.4018/978-1-60566-988-5

WHO, 2011, Assessment of health facility data quality: Data quality report card – Uganda, 2010–2011, viewed n.d., from http://www.who.int/healthinfo/country_monitoring_evaluation/UG_DataQualityReport_final_20120123.pdf

Williamson, L., Stoops, N. & Heywood, A., 2001, ‘Developing a district health information system in South Africa: A social process or technical solution?’, Studies in Health Technology and Informatics 10, 773–777.


 

Crossref Citations

1. Editorial: Measuring social impact investment
Mark Abrahams, Sibongile Walaza
African Evaluation Journal  vol: 6  issue: 2  year: 2018  
doi: 10.4102/aej.v6i2.357