An efficient health management information system (HMIS) improves health care delivery and outcomes. However, in most rural settings in Uganda, paper-based HMIS are widely used to monitor public health care services. Moreover, there are limited capabilities and capacity for quality HMIS in remote settings such as Kayunga district.
The quality assurance practices of HMIS in health centres (HCs) in Kayunga district were evaluated.
A cross-sectional descriptive study design was used to assess the quality of HMIS at 21 HCs in Kayunga district. Data were collected through in-depth interviews of HMIS focal persons as well as document analysis of HMIS records and guidelines between 15 June 2010 and 15 July 2010. The main outcomes were quality assurance practices, the HMIS programmatic challenges and opportunities. The practice of HMIS was assessed against a scale for good quality assurance practices. Qualitative data were coded and thematically analysed, whereas quantitative data were analysed by descriptive statistics using SPSS v22 software.
All the 21 HCs had manual paper-based HMIS. Less than 25% of HCs practised quality assurance measures during collection, compilation, analysis and dissemination of HMIS data. More than 50% of HCs were not practising any type of quality assurance during analysis and dissemination of data. The main challenges of the HMIS were the laborious and tedious manual system, the difficulty to archive and retrieve records, insufficient HMIS forms and difficulty in delivering hard copies of reports to relevant stakeholders influenced quality of data. Human resource challenges included understaffing where 43% of participating HCs did not have a designated HMIS staff.
The HMIS quality assurance practices in Kayunga were suboptimal. Training and support supervision of HMIS focal persons is required to strengthen quality assurance of HMIS. Implementation of electronic HMIS dashboards with data quality checks should be integrated alongside the manual system.
Remote health care settings are at risk of inadequate health care service monitoring and delivery. In the absence of established quality assurance measures for health management information systems (HMIS), the data output often has undiminished errors and may not be valued enough to be used for the intended purpose or may be misleading if used. In addition, health decisions based on poor quality data hinder effective implementation of health improvement plans (Ramesh et al.
A health management information system is a critical tool for effective and efficient health care delivery. According to the WHO, the HIS is responsible for generating, compiling, analysing and synthesising, communicating and using data for decision-making, and ensuring their overall quality. Among other functions, the information system provides information that supports patient and health facility management (WHO
Health management requires monitoring health status of the population, provision of services as to the coverage and utility, drugs stocks and consumption patterns, equipment status and availability, finances and personnel on a regular basis. This requires timely and accurate information from various sources. Health information systems in developing countries like Uganda mainly focus on epidemiological outcomes, service utilisation and finance. They generate little of the socio-cultural data needed for developing and adjusting health services and disease control programme to local health-related perceptions, values and resources.
In Uganda, the HMIS was introduced in 1997 (Hotchkiss et al.
Measures to ensure the quality of routinely collected data such as that obtained through the HMIS include allocating staff to handle data, training of the staff to enhance their knowledge and create a sense of ownership of data, provision of needed resources such as files and reporting forms, quality support supervision visits to health workers, meetings which discuss findings rather than criticise health workers, and redesigning data collection forms and procedures (Mavimbe, Braa & Bjune
There is a low demand for information because of lack of awareness by health policymakers and programme managers of the strategic importance and practical usefulness of health information for planning and management results (Bodavala). Where HMIS management is weak, data will not be used to plan, control or evaluate services rationally. Benefits of HMIS remain largely unrealised despite HMIS’s potential of being a strong health system tool (Upadhaya et al.
Health management information systems faced key challenges of poor data quality, inaccuracy and untimeliness (Selvaraju
Systems should collect only data that are needed for use. However, the information requested and collected from the primary health care and other hospitals every month is exhaustive and not warranted. Excessive information collection only burdens the system and generates into carelessness and ‘somehow to fill it and forget’ concept. The HMIS has been characterised by the absence of feedback; this defeats the purpose of information collection.
With the influx of donor funds to Uganda, some of which are tied to performance, there is a growing need for quality data to support decision-making (WHO
There are a number of constraints to implementing comprehensive HMIS with data quality assurance measures. These include lack of access to information and proper feedback by the Ministry of Health (MoH) to hospitals and lower level units; delayed feedback on the information provided by the different health institutions; lack of technical as well as managerial human capacity needed to implement a proper HMIS (Kyalo & Odhiambo-Otieno
Inadequate infrastructure, human resource and technical expertise, especially in LMIC, negatively impacts on data quality. In addition, there is a lack of effective and comprehensive systems to gather the needed health data (Eldis Health Key Issues: HMIS Uganda). Another challenge with HMIS are the behavioural and motivational factors of the people collecting and analysing the data where health care providers equate information systems with filling endless registers, collating and compiling health data, conducting minimum data analysis, and sending out reports without receiving adequate feedback (Leon, Schneider & Daviaud
The challenges are similar to those identified in other studies. A cross-country situation analysis of information systems for mental health in six LMIC found substantial policy, human resource and inadequate infrastructure, few HMIS experts, and inadequate technical support and supervision to junior staff for mental health HMIS (Upadhaya et al.
Quality of data may be assured or improved through provision of standardised reporting forms and manuals; training and quality support supervision of health facility staff who collect, analyse, report and disseminate data; meetings that discuss and validate the data presented in reports; and providing adequate feedback mechanism to the producers of data at the remote sites (Mavimbe et al.
In developing and implementing the HMIS, there is a need for strong political backing, developing a culture that values and uses information, involving all levels in changes to HMIS, starting with improving the paper-based system and ensuring that the feedback loop is continuous and reliable. There ought to be a long-term commitment to improve training and career structures for HMIS personnel working in the health and other social sectors to improve data systems. Also, improving technology and methods of measurement; improving capacity to collect and analyse data; establishing norms and standards for health measurements; and reporting valid and comparable assessments of inputs, service delivery and achievements are important aspects of monitoring and data quality assurance. New technologies can ease data collection, compilation and exchange, but clear data quality standards are still needed to ensure optimal value of HMIS data (WHO
With all the good practices, challenges and opportunities described in the preceding sections, it was not clear what practices of quality assurance of HMIS existed in Kayunga district, and to what extent any existing opportunities for data quality assurance had been exploited. It was therefore imperative that this study should be conducted to assess existing data quality assurance practices, challenges limiting implementation and ways of exploiting opportunities for data quality improvement in the district.
Evaluating quality assurance practices, challenges and opportunities for HMIS in Kayunga district.
The study assessed the extent of the data quality assurance practices as well as challenges and opportunities in implementing HMIS in Kayunga district.
The findings from the study provide information on existing data quality assurance practices, challenges and opportunities for improvement of HMIS data quality in Kayunga district, and health care services in similar facilities, and LMIC.
A user-friendly HMIS is at the core of any successful public health system and will assist hospital managers to align health system resources with the needs of service users, link performance measurements to accountability, monitor health-related activities to help assess what works and what doesn’t, and contribute to organisational development.
This study was conducted in Kayunga district among 21 public and non-government HCs. Kayunga district is located in central Uganda. It consists of two counties and three health sub-districts. There were 23 HCs (19 government and 4 private-not-for-profit) and 27 clinics including maternity centres that were not categorised under specific levels at the time of this study. Convenience sampling was performed to select Kayunga district at the time of this study. It had a good representation of HCs by level and type, located in central Uganda within close proximity to the researcher’s base to enable cost-effective data collection.
The study adopted a cross-sectional descriptive design and document analysis design. An analysis of HMIS documents and in-depth interviews of HMIS focal persons were conducted at 21 HCs in Kayunga district. Qualitative data were collected to document the quality assurance practices, challenges and opportunities for effective implementation of quality assurance practices in the district. The design explored practical solutions towards improving HMIS quality assurance practices in Kayunga district. The main study outcomes were quality assurance practices, challenges and opportunities for improvement. Qualitative data was coded and thematically analysed, whereas quantitative data were analysed using SPSS v22 software.
Data were collected through in-depth interviews using structured interview guides with questions that had been pretested for validity and reliability prior to actual data collection. Data were collected from 15 June 2010 to 15 July 2010. Primary data were obtained by interviewing health facility staff in charge of compiling HMIS data for their respective facilities. Other vital information about quality assurance practices for HMIS data at the various facilities and levels was obtained by interviewing the health facility incharge who signed off the report (HMIS 105) that the HC submitted to the district every month and the district HMIS focal person who compiled the district’s HMIS 123 report that was submitted to the MoH.
Secondary data and additional information were gathered through studying publications which included books, published and unpublished papers, Internet material, official and unofficial documents and reports.
Background characteristics of the respondents decoded included age, sex, designation, number of years of service in the position and the highest level of education. Background characteristics of the health facilities decoded included level of the facility, geographic location and type of ownership of the HC.
Data were entered in Epidata v3.1 for management. The unit of analysis was the HCs. Quantitative data were analysed by descriptive statistics using SPSS v22 software. Qualitative data were recorded in form of hand-written notes, coded, captured electronically and were manually analysed using thematic analysis to generate the themes and subthemes regarding the challenges and opportunities for implementation of HMIS.
This study was approved by the Makerere University School of Public Health as a field study. Permission to conduct the study in Kayunga district was obtained from the District Health Officer (DHO). Information from the HCs that participated in the study was recorded in such a way that it minimised disclosing the identity of the respective HC. The information gathered was kept confidential. The HC staff who participated as respondents gave informed consent following information given on the purpose, procedure, benefits and assurance of confidentiality of the information.
Response rate and HC representation was 91.3% (21/23) of the targeted HCs, with one respondent per HC. The non-response from two facilities was because of unavailability of the focal person concerned with HMIS issues at the time of the study. Majority were HC level II (52%) and public facilities (81%) (
Characteristics of health centres that participated in the study (
Facility characteristic | Category of health centre |
|||||
---|---|---|---|---|---|---|
HCII | HCIII | HCIV | HOSP | Total number | Total percentage (%) | |
Government | 7 | 7 | 2 | 1 | 17 | 81 |
Nongovernmental organisation | 4 | 0 | 0 | 0 | 4 | 19 |
None | 9 | 0 | 0 | 0 | 9 | 43 |
One | 2 | 7 | 2 | 0 | 11 | 52 |
Two | 0 | 0 | 0 | 1 | 1 | 5 |
Paper-based manual system | 11 | 7 | 2 | 1 | 21 | 100 |
< 1000 | 7 | 2 | 0 | 0 | 9 | 43 |
1000–2000 | 4 | 5 | 0 | 0 | 9 | 43 |
> 2000 | 0 | 0 | 2 | 1 | 3 | 14 |
HMIS, health management information system; HCII, health centre II, is a primary health care centre serving at a parish, i.e., village community headed by an enrolled nurse; HCIII, Health centre III, is a primary health centre that serves a sub-county and is headed by senior clinical officer; HCIV, health centre IV, is a mini-hospital that serves a constituency and is headed by a senior medical officer; HOSP, hospital, refers to Kayunga district hospital.
All 21 (100%) HCs used a manual paper-based HMIS. Less than a third (29%) of respondents understood quality assurance. The researcher sensitised respondents on the meaning of quality assurance to ensure accurate response to interview questions. A quarter (25%) of HCs practised at least one quality assurance measure during collection, compilation, analysis and dissemination of HMIS data. More than 50% of HCs did not practise any type of quality assurance during analysis and dissemination of data. Health centres and health management information system-related challenges influenced the quality of data. Specific challenges reported by respondents included manual HMIS system that was laborious and tedious, difficult to handle and retrieve records, insufficient availability of the required HMIS forms and difficulty delivering hard copies of reports to relevant stakeholders. Human resource challenges included understaffing where 43% of participating HC had no HMIS staff allocated.
The practices were categorised according to stages of data handling, that is, practices at primary data collection points, compilation, analysis, reporting and storage, and during dissemination.
All the 21 respondents practised at least one measure to ensure the quality of data that they collected (
Percentage of respondents who reported practising the stated quality assurance measure during data collection.
Less than 25% of the HCs in Kayunga had incharges check data, and 14% of the facilities reported not practising any quality assurance measure during compilation of data (
Quality assurance practices during data compilation.
More than half (52%) of the HCs in Kayunga were not practising any form of quality assurance during data analysis (
Quality assurance practices during data analysis. No single quality assurance practice mentioned for 11 (52.4%) of health centres.
Less than 25% of the HCs had their reports reviewed by the HC incharge for quality assurance before submission to the district (
Quality assurance measures practised during report compilation.
The majority (57%) of the HCs keep the data in a lockable store. However, less than 24% of the HCs practiced any one of the recommended quality assurance measures during storage of data (
Quality assurance practices implemented during data storage.
The majority (52%) of the HCs in Kayunga were not practising any quality assurance measures before disseminating HMIS data. Less than 30% of the HC practised any one of the recommended quality assurance measures during dissemination (
Reported quality assurance practices during data dissemination. No single practice mentioned for 11 (52.4%) health centres.
All 21 respondents enumerated challenges that they faced in ensuring good quality of HMIS data at their HCs of operation. All the 21 respondents mentioned at least one challenge, 14 respondents mentioned at least three challenges, while 8 respondents mentioned as many as four challenges. Thus, a sum total of 64 codes for challenges were enlisted from the 21 respondents. The challenges were thematically analysed and grouped into system-related, facility-related or human resource-related challenges (
Nature of mentioned challenges and corresponding percentage of occurrence.
Most (47%) of the challenges were system related. These included the manual HMIS that was laborious and tedious, difficult to handle and retrieve records, insufficient availability of the required HMIS forms and difficulty delivering hard copies of reports to relevant stakeholders. Some facilities mentioned improvising for HMIS forms using counter books which were ruled, but these had many inaccuracies. Also, the focal persons identified that HMIS forms did not provide options to disaggregate data for patient characteristics, for instance, there was no provision to enter the child’s gender for vitamin A and immunisation data.
Human resource (HR) challenges included understaffing where 43% of participating HCs did not have records staff; 52% of the HCs had only one records staff and only the hospital had two records staff. Only 2 of the 11 HCIIs (i.e. health centres at a parish manned by a senior nurse) had a records person, and the 2 were nongovernmental organisation (NGO)-owned facilities. As a result of understaffing, respondents stated that quality of data was usually compromised, either records were incomplete or the tallying was not performed daily as required leading to accumulation of unsummarised data. The staff handling the records, especially the HC incharges, felt incompetent to handle HMIS records because of no or limited training in records management. Also related to HR challenge was the fact that some of the staff had poor unreadable handwriting, while some did not fill all records because of laziness.
Facility-related challenges included poor handling of records where records were sometimes misplaced; lack of staff cooperation, especially when data clarification was needed; and change of health workers without proper handover. The attitude-related challenge was that staffs felt burdened by data responsibility and viewed this task as belonging to the incharge of the HCs.
The most frequently mentioned opportunities were the continuous education (28%) in form of trainings and sensitisation either during seminars or on-the-job. Other opportunities included availability of staff some of whom were computer literate and support supervision (
Opportunities for improving quality of health management information system in Kayunga district.
On whether the mentioned opportunities had been exploited, 76% of the respondents stated that the opportunities had been exploited to some extent, such as through the advance purchase of registers from the National Medical Store or early purchase and ruling of counter books for use in data capture; district organised trainings and seminars with support from some NGOs like Protecting Families Against HIV/AIDS (PREFA); district review meetings that discussed anomalies, trends and progress; and support supervision of lower HCs.
The 24% respondents who stated that opportunities had not been exploited mentioned inadequate funding; staffing norms where HCIIs did not have records staff by standard; lack of electronic databases at the HCs and training of only the records staff rather than all staff in the HC who handled data.
The study did not pose any serious risk to the respondents. Benefits of the study included insights for HCs on measures that they could use for improving and assuring quality of the information generated and disseminated through HMIS.
The findings and recommendations were based on data collected on HMIS quality assurance practices in Kayunga district at the time of the study in 2010. Since the time of the study in 2010 and the time of this publication, quality assurance practices may have changed. Nonetheless, the findings on quality assurance practices, challenges and opportunities provide insights for Kayunga district and the MoH of Uganda to improve the systems on which MoH relies for data that are used for decision-making.
The questions in the interview guide that was used for data collection were validated through pretesting on selected respondents. Thus, the findings are a close reflection of the quality assurance practices, challenges and opportunities of HMIS in Kayunga district at the time of the study.
The study aimed to assess the quality assurance practices of HMIS data in Kayunga district in Uganda. The quality assurance practices during data collection, compilation, analysis and reporting were suboptimal and required strengthening to improve quality of health care delivery and outcomes.
Firstly, we found that the quality assurance practices during collection of HMIS data, that is, use of pre-printed registers, daily or regular review of data, ensuring completeness and accuracy of data at all stages, sensitisation of staff involved in data handling, supervision of records especially by the HC incharge varied widely across HCs and were not adhered to. In addition, there was limited and inconsistent availability of data collection or reporting forms at some HCs. This impacts on quality of data collected or reported and public health implications. A similar study in Rwanda also found variability in the availability of client or patient cards in the facilities in the study districts. The same study found, however, that the majority of the health facilities in the study districts accurately transmitted data from registers to health facility monthly reports (Innocent et al.
Secondly, the quality assurance practices during data compilation also need strengthening, that is, the HC incharge checking the data, double-checks such as on the summations and aggregated data by staff responsible, immediate questioning and following up of any gaps identified, involvement of two staff, ensuring completeness and accuracy of each subset of data, regular meetings to discuss data and regular monitoring for easy detection of discrepancies. Ensuring completeness of data is a key component of quality assurance for which HCs practising it in Kayunga district are commended. This study however found limited validation or auditing of HMIS data by the HC incharge, but better than a study conducted in Pakistan which found that none of the reports reaching the District Health Office was checked for completeness (Mehmood et al.
Thirdly, during analysis, the quality practices were minimal, that is, there were limited audits by HC incharge to check and review current data, compare past and present data to detect anomalies, ensuring proper data from collection points through emphasis on keenness as well as involvement of two staff. It is recommended that routine comparisons of data are mandatory best practices in data quality management (HUD
Lastly, we also found suboptimal quality assurance practices during the HMIS report preparation. In most HCs, data were rarely double-checked; neither did the HC incharge review the report prior to dissemination, or data disaggregated and neither did the report writing involve two people to ensure high-quality data. Double-checking is a form of data quality assurance. Countries should develop manuals or tools to help the data quality checks such as the DQA procedure manual that Rwanda developed for routine DQAs at health facility level (Republic of Rwanda, Ministry of Health Rwanda
To maintain quality of data during storage, safe custody was ensured by keeping files in lockable stores or cabinets with restricted access to staff. During dissemination, staff ensured quality to avoid corruption of data or information, through reviews and discussion of the report and further checks on data. Findings from other studies in this field also revealed practices such as the use of pre-printed forms and staff allocated to data collection.
The manual HMIS was tedious and subject to many errors; HMIS forms were insufficiently available; and understaffing resulted in compromise of data as staff were tired because of work overload. Staff responsible for HMIS felt incompetent because of no or limited training in records management. Staff attitude was a challenge where HC staff felt overburdened by the record requirements, a task they viewed as the responsibility of the HC incharge. These challenges are similar to what Eldis Health Key Issues: HMIS Uganda mentioned including the fact that there was lack of an effective and comprehensive system to gather the needed health data; lack of technical and human capacity to implement a proper HMIS; non-existent link between HMIS and CME; and motivational factors of the people handling data, poor data storage and use. These challenges are similar to those reported by other studies regarding poor data quality, accuracy and timeliness (Selvaraju
Opportunities for improving quality of HMIS data included the provision of pre-printed HMIS forms; continuous education; sensitisation and training that the district organised; staff availability at all facilities, especially HCII that lacked records staff; availability of solar power which could be tapped for computer use; and regular support supervision undertaken by the district to the HCs on a quarterly basis. However, the opportunities had not been fully exploited as reported by 24% of the respondents.
Provision of HMIS manuals and pre-printed forms that served as a guide for record taking and compilation is critical for quality of data generated from such a system. The MoH HMIS manual highlighted that to ensure accuracy, written manuals describing the collection, compilation and use of the data and a training programme were to be provided. Training was also mentioned in the HMIS manual and that included both formal sessions and on-the-job follow-up. Supervision was equally critical for following up and ensuring that quality data were generated. Follow-up visits to the HC were emphasised in the HMIS manual and they had to be thorough and effective (MoH HMIS manual 2010). The WHO International Journal of Public Health (2005) mentions the need to improve training and career structures for HMIS personnel to make improvements to data systems. Murray, Lopez and Wibulpolprasert (
Mavimbe et al. (
A study in Pakistan found that health facilities were not receiving timely feedback because data were not analysed at the higher level, and this limited the improvement by the reporting facilities, with a potential consequence of poor delivery of health services (Mehmood et al.
The findings on quality assurance practices for HMIS in Kayunga district, challenges and opportunities identified provide key insights for system improvement to ensure that the data generated through the HMIS are of the highest quality possible and thus useful for decision-making.
The interpretation of the findings should be contextualised by the fact that changes may have occurred in the implementation of HMIS at HCs in Kayunga district since 2010. Nonetheless, the results provide a benchmark for HCs that use paper-based HMIS systems in remote settings in Uganda and other LMIC. Although HC staff in charge of HMIS were targeted as the primary respondents for the study, not all HCs had staff specifically designated to collect and compile HMIS data. Other staff responded in the interviews; this may have limited details available on all the quality assurance practices implemented at the HCs in the district. Related studies should collect additional data through observation of practices for a more extended time than was done in this study. Despite the mentioned quality assurance practices, the researcher did not find out the extent of data quality from the HCs; this is recommended for further research. This study was conducted in one district in Uganda because of resource constraints. Future research may be conducted in multiple districts representing all regions of Uganda for generalisation of results and recommendations.
The study revealed existence of quality assurance practices for HMIS in HCs in Kayunga, but the practices were suboptimal. The practices were quite limited as noted from the small number of HCs where the practices were implemented. The mentioned practices should be shared among HCs, implemented and replicated to a greater extent than was done at the time of this study.
Health centres faced challenges in ensuring good quality of HMIS data, ranging from system challenges to facility- and human resource-related challenges. Unless these challenges are minimised or eliminated where possible, quality of HMIS data will continually be affected.
A number of opportunities existed for improving quality of HMIS data in Kayunga district, some of which had been exploited by the district. The district should explore ways and find resources, both technically and financially to fully exploit the existing opportunities.
Training and support supervision of the HMIS focal persons are required to strengthen quality assurance of HMIS data. Implementation of electronic HMIS dashboards with data quality checks should be integrated, or it should replace the manual system to ease the laborious manual data entry and management, improve storage and ease retrieval of records at HC level.
The district should facilitate CME and training of HC staff responsible for handling data at various departments. As all staff in a way handled data, the district should facilitate training of all the staff, at least in the basic knowledge of the HMIS forms that they fill. This will ensure correct recording of data, minimise gaps and enhance completeness and accuracy of information. Health centre managers should ensure proper handover or transition from HMIS trained focal persons when staff transition from their roles.
District- and the higher-level support supervision to lower HCs should be enhanced for quality checks and improvement because these visits provide opportunities for on-the-job training. The district should consider this a priority and allocate resources.
Provision of HMIS forms in sufficient quantities is critical for enhancing quality data collection through the guiding information in pre-printed forms. The district should prioritise allocation of resources to ensure continuous availability of these forms.
Kayunga district management should encourage and facilitate regular reviews and meetings to discuss data, for example, emphasising the requirement of monthly HC staff meetings. This will provide opportunity for continuous sensitisation, iron out misunderstandings, sensitise staff to change attitude towards data and improve cooperation as staff will feel collective responsibility for their HC data. Also, meetings will enable information sharing and facilitate replication of quality assurance practices in the HCs.
The authors thank Dr Musisi Edrisa, District Health Officer of Kayunga district, for providing permission to conduct the study; respondents for their time; and HC staff for their useful insights on quality assurance practices in Kayunga district.
The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.
H.R.K. conceptualised, designed and conducted the research, entered and analysed the data and wrote the manuscript. D.K. performed additional data analysis and reviewed the manuscript for publication.