Abstract
Background: Monitoring and evaluation (M&E) is pivotal for improving the effectiveness and relevance of in-service training programmes for healthcare providers, especially in African and other low- and middle-income countries (LMICs). While information technology (IT) tools are increasingly being used to monitor and evaluate these programmes, empirical research on their application is limited.
Objectives: This systematic review aimed to critically examine and highlight the role of IT in M&E for in-service training programmes for healthcare providers in African and other LMICs.
Method: A systematic approach was undertaken, integrating information systems (IS) and evidence-based guidelines to evaluate IT tools used in M&E of in-service programmes. Studies published in English from 2014 onwards were reviewed.
Results: The review identified 28 studies meeting the inclusion criteria. Most studies – 17 out of the 28 articles (61%) – originated from Africa, 10 (36%) from Asia, and 1 (4%) from Oceania. A significant proportion of the studies – 23 out of 28 articles (82%) – reported using desktop-based software primarily for data collection, cleaning, analysis and storage.
Conclusion: The findings indicated that the increasing use of IT in the M&E of in-service training programmes for healthcare providers in LMICs holds considerable promise for improving data management and facilitating more informed decision-making to enhance healthcare delivery.
Contribution: To the best of the authors’ knowledge, this study is the first systematic review conducted to explore the use of IT tools for monitoring and evaluating in-service training programmes for healthcare providers across various health sectors in LMICs.
Keywords: information technologies; monitoring and evaluation; in-service training; healthcare providers; low- and middle-income countries; digital.
Introduction
In recent years, there has been a significant surge in the adoption and application of information and technology tools in the field of monitoring and evaluation (M&E). This trend reflects the broader digital transformation across sectors and highlights the critical role that innovative tools play in enhancing the effectiveness, efficiency and accuracy of M&E processes. Particularly within the context of training programmes, these tools offer transformative potential by addressing traditional challenges in data collection, analysis and reporting. As healthcare and treatment continue to advance rapidly, including the spread of new medical technologies, procedures and treatments worldwide, in-service training programmes also known as ‘on-the-job’ and ‘staff’ training – become essential for adopting these innovations (Nicol, Turawa & Bonsu 2019). The importance of in-service training for healthcare providers is widely recognised, although specific requirements may vary by country and organisation.
As part of lifelong learning, in-service training for healthcare providers can take various forms, including in-house workshops, seminars, webinars, hands-on practice sessions and mobile-based programmes (Mitchell et al. 2023; World Health Organization [WHO] 2013). The WHO highlights the need for continuous education and training to address the global shortage of healthcare workers and enhance healthcare services (WHO 2013). This focus on continuous education and training underscores the importance of tools used to monitor and evaluate in-service training programmes. Leveraging technology to monitor and evaluate these programmes is vital on a global scale; research has shown that it improves the accuracy and efficiency of tracking training progress, provides real-time data for informed decision-making, ensures the consistency and standardisation of training materials, and supports the implementation of effective training programmes (Buzhardt et al. 2012; Da Costa, Da Costa & Murphy 2024; Jiménez Báez et al. 2022; Nicol et al. 2019). Studies have also shown the significant impact of information technology (IT) tools in strengthening health programmes in low- and middle-income countries (LMICs), particularly in improving chronic disease management, healthcare access and adherence to treatment protocols.
Marcolino et al. (2018) emphasised the advantages of mobile health (mHealth), which involves using mobile devices such as smartphones, tablets and wearable technologies. These interventions have been associated with outcomes such as reduced mortality and hospitalisations, better glycaemic control in diabetes, improved blood pressure management in hypertensive patients and weight reduction in overweight individuals. In addition, the effectiveness of SMS reminders in boosting attendance rates and supporting adherence to tuberculosis (TB) and human immunodeficiency virus (HIV) therapies was noticed. Digital health tools such as CommCare have also demonstrated remarkable success, such as a 73% increase in antenatal care visits in India (Borkum et al. 2015). Information technology (IT) interventions for childhood illnesses have significantly outperformed traditional methods in clinical assessments and therapy accuracy (Bernasconi et al. 2024). However, despite these successes, the broader impact of digital health interventions remains inconsistent; while some tools achieved significant results, others show minimal or unmeasured effects. This inconsistency is a key challenge in using IT tools in health settings, particularly in resource-constrained environments where much health funding is allocated to vertical programmes with proven benefits, such as malaria control, polo immunisation campaigns, and HIV/acquired immunodeficiency virus (AIDS) treatment and prevention programmes.
African and other LMICs face several challenges in using technology for M&E of in-service training programmes, including limited infrastructure, such as inadequate internet connectivity and unreliable electricity, which hamper the effective deployment and utilisation of technological tools. In addition, there is often a need for more technical expertise and trained personnel to manage and maintain these systems. These factors, among others, hinder LMICs from fully leveraging technology to enhance the M&E of in-service training programmes (Nicol et al. 2019; Schoeman 2019; Stiles et al. 2021; Upadhyay, Goel & John 2023). Given the critical importance of this issue, the present systematic review aims to illuminate how IT is being utilised in the M&E of in-service training programmes for healthcare providers in LMICs.
Many articles have discussed the use of IT tools for M&E training programmes for healthcare workers in LMICs. For example, Maas et al. (2024) and Schneider et al. (2016) have focussed on cancer care. While Maas et al. (2024) explore the deployment of IT-based hospital cancer registries to track and assess cancer care in LMICs, Schneider et al. (2016) examine the implementation of IT-based hospital cancer registries in LMICs for the same purpose. This study expands upon prior work to enhance understanding within the broader healthcare context.
This article is organised into four sections. The first section outlines the methodology utilised to conduct the systematic review. The second section presents the review’s findings, emphasising the IT tools used for M&E in-service training programmes as identified by studies. The third section offers a discussion of the results, including limitations of the review, along with recommendations and suggestions for future directions. The final section discusses the implications of integrating IT tools into monitoring and evaluating practices for in-service training programmes for healthcare providers in LMICs.
Research methods and design
This systematic review draws from two sets of guidelines for systematic review research. Firstly, the guidelines developed by Okoli (2015) target information systems (IS) and are designed specifically for conducting systematic literature reviews of IS research. Secondly, this systematic review adheres to the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA), an evidence-based set of guidelines updated in 2020 (Page et al. 2021). The PRISMA guidelines provide a clear step-by-step approach for conducting systematic reviews. Okoli’s guidelines and PRISMA emphasise a systematic literature review approach, requiring a detailed search strategy and documentation of the selection process. Okoli’s guidelines provide a more detailed framework for planning and conducting a review, while PRISMA focusses more on reporting. This study benefits from combining elements of both perspectives to enhance the rigour and transparency of the systematic review.
Search strategy and inclusion criteria
Employing Okoli’s guideline, the systematic review utilised five bibliographic databases spanning multiple disciplines, including specialised databases: Web of Science, PubMed, PubMed Central, EBSCOhost and Scopus. Keywords used for the searches were related to IT (e.g., information technology OR mobile health application OR mobile phones OR online OR cloud-based OR electronic health OR information systems), M&E (e.g., monitor OR evaluate OR monitoring OR evaluating), in-service training (e.g., training programmes OR workplace training OR training workshops OR professional development OR skill enhancement OR healthcare training) and health providers (e.g., healthcare providers OR healthcare professional OR health worker OR doctors OR nursing staff OR community health worker).
To capture the latest advancements and trends in IT tools and their applications, reflecting the rapid evolution of technology and its growing integration into healthcare systems, only studies published from 2014 onwards were included. The systematic review aimed to provide a comprehensive and current overview of IT used in monitoring AND evaluating in-service training programmes in LMICs by focussing on this period. A study was included when it contained information on information technologies, mobile phones, cloud-based applications, online applications, monitoring or evaluating in-service programmes for healthcare providers and studies about in-service training for health providers, health workers, health practitioners and health professionals. A study was excluded when it did not (1) focus on in-service or ‘on-the-job’ healthcare training, (2) describe the in-service training monitoring and the evaluation process, (3) include the use of at least one IT tool, and (4) was not written in English. The World Bank classification system was used to categorise countries based on their level of economic development (Metreau, Young & Eapen 2024). Selected studies based on titles and abstracts were imported to EndNote, a citation or reference management system for inclusion and exclusion checks. Relevant studies were exported from EndNote into PICOPortal.org, an online management tool for systematic review ‘activities with customisable data extraction features for further screening and data extraction.
Data extraction
An a priori data extraction tool was developed based on the PICO (population, intervention, comparison and outcomes) elements and extracted the following from each study: type of training programme, type of technology, type of M&E process using IT and study country. The extraction tool was tested and refined prior to the start of full coding. Two authors extracted key ideas and concepts from each study regarding the use of IT for monitoring and evaluating in-service training programmes for health providers in African and other LMICs. A critical appraisal of the included evidence was not conducted because the primary aim of this research is to offer a high-level overview of the use of IT in monitoring and evaluating in-service training programmes for healthcare providers in LMICs, rather than assessing the methodological reliability of the evidence. A narrative synthesis summarised the findings and identified common patterns, themes and discrepancies across studies (Popay et al. 2006). The included studies provide information on the IT tools used to evaluate in-service training programmes for health providers.
Data synthesis
Two authors conducted thematic analysis by extracting key ideas and concepts from each included study related to IT tools used in monitoring and evaluating in-service training programmes for healthcare providers. These ideas and concepts formed themes and categories, which were arranged according to their types, the number of occurrences across the included studies and the data management aspect of the M&E process, such as data collection, data cleaning, data analysis, data reporting and data storage to identify commonalities and differences between studies. The identified commonalities and differences guided our discussion and recommendations for future IT use in monitoring and evaluating in-service training programmes for healthcare providers in LMICs.
To ensure the robustness and reliability of our findings, a third author conducted an independent search and analysis of the data through the University of Central Florida libraries. This independent verification aimed to ensure concurrence and enhance the validity of our thematic analysis, providing a well-rounded perspective on the current state and future directions of IT in M&E for healthcare training in LMICs.
Ethical considerations
This article followed all ethical standards for research without direct contact with human or animal subjects.
Results
A total of 3516 articles were identified using keywords, of which 3452 references were screened based on title. From this initial screening, 80 articles were selected for a detailed assessment based on the following criteria: (1) focus on in-service healthcare training, (2) description of training monitoring and the evaluation, (3) use of at least one IT tool and (4) written in English. After screening abstracts and titles, 35 articles were manually screened based on full text using PICO. Of these, 28 articles met the eligibility criteria and were reviewed for final inclusion in the data extraction and thematic analysis (Figure 1). Each study was assigned a unique number for easy reference throughout the rest of this review article (Table 1).
 |
FIGURE 1: Flowchart of the systematic literature review. |
|
| TABLE 1: Articles reviewed, highlighting key themes and information extracted from the articles. |
Over the past decade, there has been an increase in the number of publications on the evaluation process and tools used for in-service training for healthcare providers. Our review spanned from 2014 to 2024; however, no studies were identified prior to 2017. The studies reviewed were published between 2017 and 2024. Of the 28 studies reviewed, 9 studies (32%) were published between 2023 and 2024, while 21 studies (75%) were published in the past four years. Key themes related to the use of IT tools in the M&E of in-service training were identified, including types of IT tools, phases of the M&E process and how IT tools are utilised in different countries.
Information technology tools
Of the 28 studies, 23 (82%) reported using desktop-based software applications. Reports showed that desktop-based software programmes were used mainly for data collection, cleaning, analysis and storage (Table 2). Out of the 28 studies, 10 (35%) reported using the desktop-based statistical application Statistical Package for the Social Sciences (SPSS) for data analysis (2, 4, 9, 14, 16, 17, 19, 21, 22, 28) and 6 (21%) reported using Microsoft Excel for data collection, cleaning, analysis and storage (6, 7, 8, 15, 18, 25). Also, STATA, ATLAS.ti, Dedoose, EpiData, General Data Protection Regulation-compliant software, NVivo SAS, and ZOOM were the desktop-based applications identified in the studies.
| TABLE 2: Information technology tools used during the data collection phase of monitoring and evaluation processes. |
A total of 11 (39%) of the 28 studies used web-based applications for data collection and storage. These applications were often referred to in general terms such as servers (20, 23, 27), online questionnaires (1, 11, 15, 23), online platforms (4, 9, 15, 24) and other web-based platforms (7, 21) (Table 2). Furthermore, 9 (32%) studies employed mobile-based applications and devices such as mobile devices, tablets and phones (1, 10, 11, 20, 26, 27), WhatsApp (14), InStrat app (23) and the Mobile Academy database (3). Four (14%) of the 28 studies used learning platforms for in-service training, through which evaluation data on the training programme were collected. In addition, 2 (7%) studies used digital health records systems for data collection and storage, while 6 (21%) studies utilised peripheral devices, such as audio recorders (Table 2 and Table 3). While 1 (4%) study indicated the use of IT, such as electronic health record systems in the data reporting stage (11), 2 (7%) studies referenced the use of digital health record systems (9, 11) in data collection stage. These systems were integrated into training programmes, allowing healthcare providers to learn and practise using them.
| TABLE 3: Type of information technology used during data collection, analysis, storage, and reporting phases of monitoring and evaluation processes. |
The most common types of IT were desktop-based software applications, web-based applications, mobile-based applications and devices, learning management systems (LMS) and health management systems (Table 2). Learning management systems platforms were utilised for both training delivery and data analysis. In Ethiopia, an LMS was integrated to evaluate the effectiveness of the 10-module blended learning course on TB diagnosis for laboratory professionals in a resource-limited setting (12). Few studies used health management systems for M&E purposes, such as data collection and reporting (9, 11).
The review found the use of multiple IT tools across at least three of the five phases (i.e., data collection, data cleaning, data analysis, data storage and reporting) of the M&E process (10, 11, 18, 23, 25, 27) in the articles. Of these, only one study (11) reportedly used IT tools in five phases of the M&E process. In a study on in-service training for forward-based primary healthcare outreach teams by Mantell et al. (2022), IT was employed in data collection, storage, cleaning, analysis and reporting.
Interview sessions were digitally recorded and a 15-min online survey was administered using the Qualtrics tool. For data management and analysis, qualitative data were entered and coded using the DedooseTM Software Package. In contrast, quantitative data from the health facility assessments, knowledge, attitude, and practice (KAP) surveys, and field observations were collected on tablets and uploaded to a central SurveyCTOTM server. The data were downloaded from SurveyCTO (Dobility Inc., Cambridge, Massachusetts) and Qualtrics (Qualtrics International Inc., Seattle, Washington), cleaned and analysed using Statistical Analysis System (SAS) (SAS Institute Inc., Cary, North Carolina). System-generated reports were obtained from the electronic mHealth platform. Most studies employed paper-based, manual and electronic or online approaches.
Only 1 (4%) of the 28 studies indicated the benefits of IT in evaluating in-service training, particularly in data collection of the M&E process (15). Otu et al. (2021) highlighted that evaluating learning and skills acquisition through the same InStrat platform ensures that training participants can receive in-service training with minimal disruption (23). The other 27 studies did not address the benefits.
Only 1 of the 28 studies discussed the challenges or recommendations for using IT tools while evaluating the in-service training programmes. In terms of the challenges of using IT in the evaluation of in-service training in LMICs and how they were resolved, only 1 study (12) mentioned difficulty in collecting data using IT from training participants who were not able to get the Internet connectivity needed to access the IT tools, and as a result, could not respond on the training been monitored and evaluated. These findings led the authors (12) to recommend strategies such as providing free Internet access at health facilities and implementing training cost reimbursement programmes to support participants better. The other 27 reviewed studies did not include any discussion of the challenges related to using IT for M&E. In addition, the reviewed studies did not address topics such as IT infrastructure, capacity building, data security, or the use of IT-based knowledge graphs, which are tools that organise and visualise complex data relationships, in the M&E processes.
Monitoring and evaluation processes
Different M&E activities were used. The ‘Evaluation’ activity of the M&E process is the most common activity among the 28 articles reviewed. The review found several in-service training programmes developed, implemented, monitored and evaluated using IT tools across different countries. In terms of steps or procedures for evaluating the in-service training programme, the following were common across the studies reviewed: use of an evaluation framework or model, such as the Kirkpatrick training evaluation model and process evaluation (1, 3, 6, 9, 17, 18, 20, 24, 26); the use of knowledge assessment such as pre and post assessments (8, 11, 14, 21, 27); and the 3-month and 6-month follow-up assessments of the training participants (19, 25). The majority of the studies used IT for data analysis (24 out of 28 articles, or 86%) and the data collection phase (19 out of 28 articles, or 68%) of the M&E process, with fewer studies mentioning the use of IT tools for reporting (1 out of 28 articles, or 4%) (Table 1). These IT tools for the data analysis stage of the evaluation process allow the authors of the reviewed studies to generate findings and give interpretations of the data obtained. Some tools enable statistical calculations on M&E findings, such as summing data, data calculating averages, comparing knowledge assessments, conducting group tests and creating data visualisations such as tables.
Furthermore, M&E indicators, including knowledge assessment scores, system log records and course completions, were monitored with data collected when IT tools such as the LMS were used. The IT tools eased the monitoring activities of the training. However, a few of the studies that employed LMS in the training programme mentioned that some of the training participants could access the LMS, which limits the amount of data that could be obtained about the training. For example, Manyazewal et al. (2017) identified course completion as one of the M&E indicators for in-service training, and the LMS was used to track this data (12). Although a large number of participants completed the course, the remaining participants could not be tracked because of infrastructure issues, such as inconsistent internet connectivity. Insights from the LMS led the authors to recommend solutions to support the participants, free internet access at health facilities and proposing training cost reimbursement programmes. None of the studies reviewed examined data security and privacy concerns.
Characteristics of countries
Of the 28 studies, 21% (6) were from Nigeria and 14% (4) were from Iran (Figure 2). Each of the following countries had two studies (7%): China, Ethiopia, South Africa, Turkey and Uganda. Other countries represented by at least one study include: India, Kenya, Malawi, Papua New Guinea, Sudan, Tanzania and Vietnam. The in-service training programmes targeted various healthcare providers, such as nurses, community health workers, midwives, doctors and medical laboratory personnel. In 96% (27) of the studies, in-service training programmes were primarily conducted at facility, district, regional and national levels. According to the World Bank’s classification, while 21 studies (75%) were carried out in middle-income countries (2–7), 6 (21%) (8–28) were conducted in low-income countries. Studies from low-income countries mainly focussed on the use of IT tools for data collection (2, 3, 5, 6) and data analysis (2, 3, 4, 6, 7) phases of the evaluation process. In contrast, studies from middle-income countries discussed IT tools across various phases of the evaluation process. Notably, in low-income countries, IT tools were not mentioned for data cleaning, data storage and reporting phases of the M&E process for in-service training programmes for health providers. Similarly, in middle-income countries, few mentions were made regarding the use of IT tools for data cleaning, data storage and reporting phases of the M&E process for these training programmes (Table 1).
 |
FIGURE 2: Number of reviewed studies per country. |
|
Discussion
This systematic review highlighted the critical role of IT tools in enhancing the M&E of in-service training programmes for healthcare providers in African and other LMICs. The findings reveal extensive use of technologies such as desktop-based software, mobile-based applications, LMS and electronic health records. These tools significantly improve the efficiency and responsiveness of M&E processes by enabling the real-time data collection, analysis and feedback, which is particularly valuable in resource-limited settings. Despite these advantages, the review underscores gaps in outcome-level evaluations of IT tools compared to traditional methods, signalling the need for further comparative research. The results also emphasise the challenges of implementing IT tools in LMICs, including inadequate infrastructure, inconsistent internet connectivity and limited technical expertise among healthcare providers. These barriers hinder the full utilisation of IT tools across all phases of the M&E process, particularly in data reporting and storage. Addressing these issues requires targeted investments and innovations tailored to the unique needs of LMICs.
Another significant gap identified is the need for more focus on data security and privacy concerns in the reviewed studies. Given the potential sensitivity of healthcare and training data, ensuring robust data protection measures is critical. In addition, the review highlights the limited integration of IT tools into all M&E phases, with most studies concentrating on data collection and analysis. This underutilisation points to missed opportunities for a comprehensive evaluation of in-service training programmes. The reviewed studies show promising applications of mobile-based devices, especially in remote and underserved areas. Mobile tools enable real-time data collection and processing, providing timely feedback and decision-making capabilities. However, the lack of focus on infrastructure challenges, particularly in LMICs, limits these tools’ broader applicability and scalability. The findings call for more longitudinal and comparative studies to assess the long-term impact of IT tools on healthcare outcomes and professional development in LMICs.
A major strength of this systematic review lies in its extensive inclusion of studies spanning diverse African countries, LMIC regions and a variety of healthcare settings. The study’s application of two systematic review guidelines enhanced its rigour, providing empirical insights into the IT tools employed in M&E processes for in-service training programmes. Nonetheless, there are important limitations to this review. The exclusion of non-English studies and grey literature may have narrowed the scope of findings and potentially overlooked valuable insights from non-English speaking regions or unpublished reports. Moreover, the quality of assurance in the reviewed studies was considered moderate, as many focussed more on the M&E process than on the IT tools themselves, potentially leading to underreporting of the challenges and benefits of using IT tools in these settings.
Recommendations
The findings from this review underscore the importance of targeted strategies to enhance the effectiveness of IT tools in the M&E of in-service training programmes for healthcare providers in African and other LMICs. Addressing the infrastructural challenges identified, such as unreliable Internet and power supply, is a critical priority. Studies such as Manyazewal et al. (2017) emphasise the need for robust investments in reliable connectivity and hardware to support IT-enabled M&E processes. Such investments should focus on enhancing interoperability between systems to enable seamless data exchange, improving efficiency and reducing redundancy.
Capacity building is another vital area of focus. The review highlights limited training opportunities for healthcare providers in using IT tools effectively. Otu et al. (2021) demonstrate that incorporating hands-on training into in-service programmes fosters practical utility and boosts user confidence. Training programmes should be iterative, with regular updates to ensure that healthcare professionals remain adept at leveraging advancements in IT tools. This approach aligns with Kunnibe (2023), who advocate for continuous professional development to support the adoption of digital health technologies.
Data security and privacy concerns, although underexplored in the reviewed studies, are critical given the sensitive nature of healthcare and training data. Agarwal, Alhuwail and John (2021) emphasise the importance of robust protocols, including encryption, secure storage and access controls, to safeguard data integrity. Developing comprehensive policies and regulations specific to LMIC contexts will further strengthen data protection measures.
There is a pressing need for comparative research to evaluate the effectiveness of IT-based M&E tools relative to traditional methods. Mantell et al. (2022) demonstrated the potential of IT tools to streamline M&E processes across multiple phases, but systematic comparisons would provide evidence for broader adoption. Fraser et al. (2023) also underscore the value of comparative studies in identifying best practices for digital health interventions in resource-constrained settings.
Tailored innovations that address local challenges are essential for the successful implementation of IT tools. Manyazewal et al. (2017) highlight the importance of offline functionality in tools such as SurveyCTO to overcome connectivity barriers in Ethiopia. Partnerships between governments, healthcare institutions, universities, non-profits and IT developers can drive the development of user-friendly, contextually relevant solutions that align with the unique needs of LMICs, as suggested by Otu et al. (2021).
Integrating advanced technologies such as artificial intelligence (AI) and knowledge graphs into M&E processes has transformative potential (Fraser et al. 2023). Artificial intelligence applications can analyse large datasets, predict outcomes and optimise resource allocation, while knowledge graphs facilitate the visualisation of complex data relationships, aiding decision-making. Fraser et al. (2023) point to the scalability and adaptability of these technologies in LMIC healthcare settings, making them valuable tools for enhancing M&E processes.
Mobile-based applications and LMS platforms, which showed significant promise in this review, warrant targeted investments. Mobile devices are particularly effective in remote and underserved areas, enabling real-time data collection and feedback. Otu et al. (2021) illustrate the utility of mobile tools in improving training programme evaluation in Nigeria. Similarly, Manyazewal et al. (2017) demonstrate that LMS platforms can enhance training programme tracking and reporting, even in resource-limited settings.
Longitudinal studies are needed to evaluate the sustainability and long-term impact of IT tools on healthcare delivery and professional development. The emerging nature of this field, with 32% of reviewed studies published in 2023–2024, underscores the importance of ongoing research. Kunnibe (2023) highlight the value of longitudinal evaluations in understanding the broader implications of digital health interventions.
Finally, addressing infrastructure gaps is crucial for equitable access to IT tools. Innovative solutions could mitigate barriers, such as providing free Internet access at healthcare facilities and offering reimbursement programmes for training costs.
Conclusion
In conclusion, this review underscores the critical need for further research into the long-term impacts of IT tools specifically designed for monitoring and evaluating healthcare professional development and service delivery in African and other LMICs. Longitudinal and comparative studies are essential to strengthening the evidence base, enabling informed policy decisions that enhance M&E practices and improve healthcare outcomes in resource-constrained settings.
Acknowledgements
The authors would like to thank Nelson Mandela University, Temple University, and the University of Central Florida for supporting the importance of this research area.
Competing interests
The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.
Authors’ contributions
Y.O., D.v.G. and L.D.I. designed the study. Y.O., D.v.G., O.M. and L.D.I. participated in the search strategy of the systematic review process. Y.O. analysed the data and generated results. D.v.G, L.D.I. and O.M. gave input on the analysis and on the results of the analysis. Y.O., D.v.G., O.M. and L.D.I wrote, reviewed, and edited the manuscript.
Funding information
This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Data availability
The data that support the findings of this study are available from the corresponding author, Y.O., upon reasonable request.
Disclaimer
The views and opinions expressed in this article are those of the authors and are the product of professional research. The article does not necessarily reflect the official policy or position of any affiliated institution, funder, agency or that of the publisher. The authors are responsible for this article’s results, findings and content.
References
Agarwal, S., Alhuwail, D. & John, O., 2021, ‘mHealth solutions in global health applications and implementation in low- and middle-Income countries’, in I. Kickbusch, D. Ganten & M. Moeti (eds.), Handbook of global health, pp. 1747–1774, Springer International Publishing, Cham.
Ahmed, W., Abdelrahim, M., Gloyd, S., Farquhar, C. & Puttkammer, N., 2024, ‘Evaluating the long-term impact of large-scale trainings: An exposure based cross-sectional study on female genital mutilation-related knowledge, attitudes and practices among Sudanese midwives in Khartoum State’, BMJ Open 14(1), e076830. https://doi.org/10.1136/bmjopen-2023-076830
Arzani, A., Valizadeh, S., Poorkaremi, S., Taheri Ezbarami, Z. & Ghojazadeh, M., 2020, ‘Evaluating the impact of a multimedia training versus lecture training on attitudes and practices in paediatric nurses in children pain management: A randomised controlled trial’, Nurs Open 7(4), 1032–1038. https://doi.org/10.1002/nop2.476.
Bashingwa, J.J.H., Shah, N., Mohan, D., Scott, K., Chamberlain, S., Mulder, N. et al., 2021, ‘Examining the reach and exposure of a mobile phone-based training programme for frontline health workers (ASHAs) in 13 states across India’, BMJ Global Health 6(suppl 5), e005299. https://doi.org/10.1136/bmjgh-2021-005299
Bernasconi, A., Landi, M., Yah, C.S. & Van Der Sande, M.A., 2024, ‘Information and communication technology to enhance the implementation of the integrated management of childhood illness: A systematic review and meta-analysis’, Mayo Clinic Proceedings: Digital Health 2(3), 438–452. https://doi.org/10.1016/j.mcpdig.2024.06.005
Borkum, E., Sivasankaran, A., Sridharan, S., Rotz, D., Sethi, S., Manoranjini, M. et al., 2015, Evaluation of the Information and Communication Technology (ICT) Continuum of Care Services (CCS) intervention in Bihar, Mathematica Policy Research Reports, viewed 15 August 2024, from https://www.mathematica.org/our-publications-and-findings/publications/evaluation-of-the-information-and-communication-technology-ict-continuum-of-care-services-ccs.
Buzhardt, J., Walker, D., Greenwood, C.R. & Heitzrnan-Powell, L., 2012, ‘Using technology to support progress monitoring and data-based intervention decision making in early childhood: Is there an app for that?’, Focus on Exceptional Children 44(8), 1–18. https://doi.org/10.17161/foec.v44i8.6914
Da Costa, T.P., Da Costa, D.M.B. & Murphy, F., 2024, ‘A systematic review of real-time data monitoring and its potential application to support dynamic life cycle inventories’, Environmental Impact Assessment Review 105, 107416. https://doi.org/10.1016/j.eiar.2024.107416
Farokhzadian, J., Farahmandnia, H., Tavan, A., Taskiran Eskici, G. & Soltani Goki, F., 2023, ‘Effectiveness of an online training program for improving nurses’ competencies in disaster risk management’, BMC Nursing 22, 334. https://doi.org/10.1186/s12912-023-01497-1
Ferla, J.P., Gill, M.M., Komba, T., Abubakar, A., Remes, P., Jahanpour, O. et al., 2023, ‘Improvement of community health worker counseling skills through early childhood development (ECD) videos, supervision and mentorship: A mixed methods pre-post evaluation from Tanzania’, PLOS Global Public Health 3(6), e0001152. https://doi.org/10.1371/journal.pgph.0001152
Firooznia, M., Hamta, A. & Shakerian, S., 2020, ‘The effectiveness of in-service training “pharmacopeia home health” based on Kirkpatrick’s model: A quasi-experimental study’, Journal of Education and Health Promotion 9(1), 218. https://doi.org/10.4103/jehp.jehp_170_20
Fraser, H.S., Marcelo, A., Kalla, M., Kalua, K., Celi, L. A. & Ziegler, J., 2023, ‘Digital determinants of health: Editorial’, PLOS Digital Health 2(11), e0000373. https://doi.org/10.1371/journal.pdig.0000373
Jiménez Báez, M.V., Gutiérrez De La Cruz, M.E., Chávez Hernández, M.M., Martínez Castro, L.R. & Nuñez, F.J.A., 2022, ‘Digital quality resources resulting from standardized program for rubric training in medical residents’, Healthcare 10(11), 2209. https://doi.org/10.3390/healthcare10112209
Kessy, S.J., Gon, G., Alimi, Y., Bakare, W.A., Gallagher, K., Hornsey, E. et al., 2023, ‘Training a continent: A process evaluation of virtual training on infection prevention and control in Africa during COVID-19’, Global Health: Science and Practice 11(2), e2200051. https://doi.org/10.9745/GHSP-D-22-00051
Kunnibe, N., 2023, ‘Using routine data for long term impact evaluation: Methodological reflections from a complex health system intervention in a low-income context’, Cogent Public Health 10(1), 2153448. https://doi.org/10.1080/27707571.2022.2153448
Legesse, S., Alemu, T., Tassew, M., Shiferaw, B., Amare, S., Tadesse, Z. et al., 2020, ‘Evaluation of in-service training program of laboratory professionals in Amhara Public Health Institute Dessie Branch, northeast Ethiopia: A concurrent mixed-method study’, PLoS One 15(12), e0243141. https://doi.org/10.1371/journal.pone.0243141
Lin, H., Liu, G., Wang, X., Xu, Q., Guo, S. & Hu, R., 2023, ‘A virtual simulation-based training program on birthing positions: A randomized controlled trial’, BMC Nursing 22, 318. https://doi.org/10.1186/s12912-023-01491-7
Little, K.M., Nwala, A.A., Demise, E., Archie, S., Nwokoma, E.I., Onyezobi, C. et al., 2023, ‘Use of a hybrid digital training approach for hormonal IUD providers in Nigeria: Results from a mixed method study’, BMC Health Services Research 23, 1316. https://doi.org/10.1186/s12913-023-10211-5
Maas, M.R., Yang, A., Muir, M.A., Collins Iv, J.B., Canter, C., Tamamyan, G. et al., 2024, ‘Evaluating implementation of a hospital-based cancer registry to improve childhood cancer care in low- and middle-income countries’, Cancer Medicine 13(17), e70125. https://doi.org/10.1002/cam4.70125
Mantell, J.E., Masvawure, T.B., Zech, J.M., Reidy, W., Msukwa, M., Glenshaw, M. et al., 2022, ‘“They are our eyes outside there in the community”: Implementing enhanced training, management and monitoring of South Africa’s ward-based primary healthcare outreach teams’, PLoS One 17(8), e0266445. https://doi.org/10.1371/journal.pone.0266445
Manyazewal, T., Marinucci, F., Belay, G., Tesfaye, A., Kebede, A., Tadesse, Y. et al., 2017, ‘Implementation and evaluation of a blended learning course on tuberculosis for front-line health care professionals’, American Journal of Clinical Pathology 147(3), 285–291. https://doi.org/10.1093/ajcp/aqx002
Marcolino, M.S., Oliveira, J.A.Q., D’agostino, M., Ribeiro, A.L., Alkmim, M.B.M. & Novillo-Ortiz, D., 2018, ‘The impact of mHealth interventions: Systematic review of systematic reviews’, JMIR mHealth and uHealth 6(1), e23. https://doi.org/10.2196/mhealth.8873
Mastellos, N., Tran, T., Dharmayat, K., Cecil, E., Lee, H.-Y., Wong, C.C.P. et al., 2018, ‘Training community healthcare workers on the use of information and communication technologies: A randomised controlled trial of traditional versus blended learning in Malawi, Africa’, BMC Medical Education 18, 1–13. https://doi.org/10.1186/s12909-018-1175-5
Metreau, E., Young, E.K. & Eapen, S.G., 2024, World Bank country classification by income level for 2024–2025, viewed 14 June 2014, from https://blogs.worldbank.org/en/opendata/world-bank-country-classifications-by-income-level-for-2024-2025.
Mitchell, S., Phaneuf, J.C., Astefanei, S.M., Guttormsen, S., Wolfe, A., De Groot, E. & Sehlbach, C., 2023, ‘A changing landscape for lifelong learning in health globally’, Journal of CME 12(1), 1–4. https://doi.org/10.1080/21614083.2022.2154423
Moeteke, N.S., Oyibo, P., Ochei, O., Ntaji, M.I., Awunor, N.S., Adeyemi, M.O. et al., 2024, ‘Effectiveness of online training in improving primary care doctors’ competency in brief tobacco interventions: A cluster-randomized controlled trial of WHO modules in Delta State, Nigeria’, PLoS One 19(2), e0292027. https://doi.org/10.1371/journal.pone.0292027
Mohamed, Y., Hezeri, P., Kama, H., Mills, K., Walker, S., Hau’ofa, N. et al., 2023, ‘Evaluation of an online training program on COVID-19 for health workers in Papua New Guinea’, Tropical Medicine and Infectious Disease 8(6), 327. https://doi.org/10.3390/tropicalmed8060327
Momennasab, M., Mohammadi, F., Dehghanrad, F. & Jaberi, A., 2023, ‘Evaluation of the effectiveness of a training programme for nurses regarding augmentative and alternative communication with intubated patients using Kirkpatrick’s model: A pilot study’, Nursing Open 10(5), 2895–2903. https://doi.org/10.1002/nop2.1531
Munezero, J.B.T., Atuhaire, C., Groves, S. & Cumber, S.N., 2018, ‘Assessment of nurses’ knowledge and skills following cardiopulmonary resuscitation training at Mbarara Regional Referral Hospital, Uganda’, Pan African Medical Journal 30(1), 108. https://doi.org/10.11604/pamj.2018.30.108.15398
Mutambo, C., Shumba, K. & Hlongwana, K.W., 2020, ‘Post-training and mentorship experiences of KidzAlive-trained healthcare workers at primary healthcare facilities in KwaZulu-Natal, South Africa’, African Journal of Primary Health Care & Family Medicine 12(1), 1–11. https://doi.org/10.4102/phcfm.v12i1.2109
Nicol, E., Turawa, E. & Bonsu, G., 2019, ‘Pre- and in-service training of health care workers on immunization data management in LMICs: A scoping review’, Human Resources for Health 17(92), 1–14. https://doi.org/10.1186/s12960-019-0437-6
Nwankwo, B. & Sambo, M.N., 2018, ‘Can training of health care workers improve data management practice in health management information systems: A case study of primary health care facilities in Kaduna State, Nigeria’, Pan African Medical Journal 30(1), 289, 1–8, viewed 11 June 2024, from https://www.ajol.info/index.php/pamj/article/view/183301/172666.
Nwokoma, E., Anyasi, H., Archie, S., Onyezobi, C., Olaolorun, F., Anyanti, J. et al., 2023, ‘Use of Objective Structured Clinical Examination (OSCE) in a hybrid digital/in-person training for hormonal IUD in Nigeria: Findings and applications of the approach’, Gates Open Research 7, 120. https://doi.org/10.12688/gatesopenres.14695.1
Odabasi, O., Elcin, M., Uzun Basusta, B., Gulkaya Anik, E., Aki, T.F. & Bozoklar, A., 2018, ‘Development and evaluation of a training program for organ procurement coordinators using standardized patient methodology’, Experimental and clinical transplantation 16, 481–487. https://doi.org/10.6002/ect.2015.0131
Okoli, C., 2015, ‘A guide to conducting a standalone systematic literature review’, Communications of the Association for Information Systems 37, 879–910. https://doi.org/10.17705/1CAIS.03743
Okuroğlu, G.K. & Alpar, S.E., 2019, ‘Effect of web-based diabetes training program on diabetes-related knowledge, attitudes, and skills of health professionals: A randomized controlled trial’, Japan Journal of Nursing Science 16(2), 184–193. https://doi.org/10.1111/jjns.12228
Otu, A., Okuzu, O., Effa, E., Ebenso, B., Ameh, S., Nihalani, N. et al., 2021, ‘Training health workers at scale in Nigeria to fight COVID-19 using the InStrat COVID-19 tutorial app: An e-Health interventional study’, Therapeutic Advances in Infectious Disease 2021(8), 1–13. https://doi.org/10.1177/20499361211040704
Page, M.J., Mckenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D. et al., 2021, ‘The PRISMA 2020 statement: An updated guideline for reporting systematic reviews’, British Medical Journal 372, n71. https://doi.org/10.1136/bmj.n71
Popay, J., Roberts, H., Sowden, A., Petticrew, M., Arai, L., Rodgers, M. et al., 2006, Guidance on the conduct of narrative synthesis in systematic reviews, A product from the ESRC methods programme, viewed 11 June 2024, from https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=ed8b23836338f6fdea0cc55e161b0fc5805f9e27.
Sabin, L.L., Mesic, A., Le, B.N., Halim, N., Cao, C.T.H., Bonawitz, R. et al., 2022, ‘Costs and cost-effectiveness of mCME Version 2.0: An SMS-based continuing medical education program for HIV clinicians in Vietnam’, Global Health, Science and Practice 10(4), e2200008. https://doi.org/10.9745/GHSP-D-22-00008
Schneider, M., Van De Water, T., Araya, R., Bonini, B., Pilowsky, D., Pratt, C. et al., 2016, ‘Monitoring and evaluating capacity building activities in low and middle -income countries: Challenges and opportunities’, Global Mental Health 3, e29. https://doi.org/10.1017/gmh.2016.24
Schoeman, F., 2019, ‘Digital tools for training frontline health workers in low and middle-income countries: A systematic review’, Master’s thesis, Faculty of Health Sciences, University of Cape Town.
Shikuku, D.N., Nyaoke, I., Maina, O., Eyinda, M., Gichuru, S., Nyaga, L. et al., 2022, ‘The determinants of staff retention after emergency obstetrics and newborn care training in Kenya: A cross-sectional study’, BMC Health Services Research 22(1), 872. https://doi.org/10.1186/s12913-022-08253-2
Stiles, C.E., O’Neil, E. Jr., Kabali, K. & O’Donovan, J., 2021, ‘The use of low-cost ruggedized Android tablets to augment in-service training of community health workers in Mukono, Uganda: Perspectives and lessons learned from the field’, African Health Sciences 21(3), 1482–1490. https://doi.org/10.4314/ahs.v21i3.60
Ugwa, E., Kabue, M., Otolorin, E., Yenokyan, G., Oniyire, A., Orji, B. et al., 2020, ‘Simulation-based low-dose, high-frequency plus mobile mentoring versus traditional group-based trainings among health workers on day of birth care in Nigeria: A cluster randomized controlled trial’, BMC Health Services Research 20, 586. https://doi.org/10.1186/s12913-020-05450-9
Upadhyay, K., Goel, S. & John, P., 2023, ‘Developing a capacity building training model for public health managers of low and middle income countries’, PLoS One 18(4), e0272793. https://doi.org/10.1371/journal.pone.0272793
Wang, F., Xiao, L.D., Wang, K., Li, M. & Yang, Y., 2017, ‘Evaluation of a WeChat-based dementia-specific training program for nurses in primary care settings: A randomized controlled trial’, Applied Nursing Research 38(1), 51–59. https://doi.org/10.1016/j.apnr.2017.09.008
World Health Organization (WHO), 2013, Transforming and scaling up health professionals’ education and training: World Health Organization guidelines 2013, World Health Organization, viewed 05 June 2024, from https://iris.who.int/
|
Hide
Show all
Email this article (Login required)
