About the Author(s)


Steven Masvaure symbol
Centre for Learning on Evaluation and Results-Anglophone Africa, Faculty of Commerce, Law and Management, University of the Witwatersrand, Johannesburg, South Africa

Tebogo E. Fish Email symbol
Centre for Learning on Evaluation and Results-Anglophone Africa, Faculty of Commerce, Law and Management, University of the Witwatersrand, Johannesburg, South Africa

Citation


Masvaure, S. & Fish, T.E., 2022, ‘Strengthening and measuring monitoring and evaluation capacity in selected African programmes’, African Evaluation Journal 10(1), a635. https://doi.org/10.4102/aej.v10i1.635

Original Research

Strengthening and measuring monitoring and evaluation capacity in selected African programmes

Steven Masvaure, Tebogo E. Fish

Received: 12 Apr. 2022; Accepted: 24 Oct. 2022; Published: 15 Dec. 2022

Copyright: © 2022. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background: Strengthening the capacities of countries and organisations to perform monitoring and evaluation (M&E) functions is gaining momentum in the Global South. However, there is limited literature on the effectiveness and impact of these capacity strengthening initiatives in Africa. Across the continent, there has been a global push to strengthen M&E capacity both within the state and non-state sector. The rationale for the push and investments is based on the premise that M&E capacity is critical for assisting public officials, non-state sector development managers, non-governmental organisations, and donors to improve the design and implementation of their projects, improve progress, increase impact, and enhance learning. Despite considerable investments to build M&E capacity in the African context, literature shows that the measurement of these initiatives is non-existent.

Objectives: To explore M&E capacity strengthening initiatives and how their effectiveness is being measured.

Method: The study adopted a qualitative research approach, specifically using semi-structured interviews to gain an in-depth understanding of capacity-strengthening approaches and how capacity strengthening activities are measured. A sample was drawn from Botswana, Ethiopia, Ghana, Kenya, Rwanda, Tanzania, Uganda, and Zambia.

Results: This study found that M&E capacity strengthening in the selected countries is ad hoc, indiscriminate, haphazard and mainly focuses on developing individual skills and abilities.

Conclusion: The significance of strengthening M&E system capacity in Anglophone Africa has been strongly supported by this study, considering the critical impact that effective M&E systems have in enabling countries to reach their development goals.

Keywords: monitoring and evaluation (M&E); capacity-strengthening; training; measurement; system-level; institutional level; individual level.

Introduction

In this article, monitoring and evaluation (M&E) capacity strengthening is defined as the process of improving the ability of individuals, institutions and the overall M&E system to meet its M&E objectives and expectations (Tarsilla 2014). We take cognisance of the debate and negative connotations associated with capacity development and capacity building. Monitoring and evaluation capacity building refers to the building of new M&E systems according to a prescribed design, while capacity development refers to building on existing M&E skills and knowledge, driving a dynamic and flexible process of change, borne by local actors (Tarsilla 2014). In this article, we collectively refer to the terms capacity building and capacity development as capacity strengthening (CS), which we are defining as the process through which individuals, institutions and countries develop, enhance and organise their M&E systems to effectively monitor the performance and evaluate development interventions at both state and institutional (non-state or state) levels (Masvaure, Kiwekete & Chirau 2020).

In this article, M&E capacity is viewed as consisting of three interwoven levels: individual, organisational or institutional and system or country levels. Individual capacity level refers to individual technical skills and abilities (Babu 2018), while at an institutional level, capacity refers to having systems that enable M&E functions within organisation, evidence of such systems is revealed through processes, structures, governance and management frameworks, resources and processes (Struyk, Damon & Haddaway 2011). At the system level, coherent policies, culture, structures, behaviour and strategies among state and non-state entities create an enabling environment for the development and use of M&E system information and evidence (Nuyens 2005).

We view M&E capacity strengthening as a long-term change process, aimed at strengthening capacity in related management systems, developing national or sectoral performance M&E systems. It is also a means to support more effective policies and programmes as well as help achieve high levels of public and private sector performance (Morkel & Ramasobama 2017). On the other hand, M&E capacity strengthening involves the design and implementation of teaching and learning strategies to help individuals, groups and organisations learn about what constitutes effective, useful and professional M&E practice (Morkel & Ramasobama 2017). It reflects an intention to increase participants’ knowledge and skills and to generate more positive attitudes towards M&E (Preskill & Boyle 2008). Sustainable M&E practice also requires the development of systems, processes, policies and plans that help embed M&E work into the way organisations and governments accomplish their mission and strategic goals, the ultimate being the use of information and evidence from the M&E system in decision-making (Lopez-Acevedo et al. 2012).

Importance of monitoring and evaluation capacity strengthening in the development space

Monitoring and evaluation are integral, yet distinct aspects of the development programme life cycle (Görgens & Kusek 2010). They are both critical mechanisms for forward-looking strategic positioning, institutional learning and sound management. Monitoring and evaluation are intended to influence decision-making, including decisions to improve, re-orient or discontinue an evaluated intervention or policy; decisions about wider organisational strategies or management structures and decisions by national and international policymakers and funding agencies (Kemerer & Salentine 2014). As a result, M&E capacity strengthening has grown immensely over the years because of the realisation that development interventions and programmes need to be informed by good evidence of what works and an understanding of why it works (Basheka & Byamugisha 2015). It has also grown because of the increase in the demand for M&E by national and international donors, government agencies and others (Morkel & Ramasobama 2017).

There is a myriad of reasons why individuals, institutions and countries engage in M&E capacity strengthening. First, this is done to meet accountability requirements and for them to be more effective or competitive in seeking new and/or increased funding (Farrell 2009). Second, there is often a commitment to learning from M&E that focuses on programme improvement, developing a culture of investigation and institutional effectiveness (Woodhill 2019). Third, there is a desire to communicate more effectively about a programme’s process, progress and impact with external audiences (Weiss 1998). Fourth, there is a need to distribute M&E skills, knowledge and attitudes throughout an institution or multiple programmes and sites (Preskill 2008). Fifth, changes in the external environment, such as the national economy, increased acceptance of evaluation nationwide, new political appointees, etc. Sixth, there may be a desire to enlighten and support government policymaking and planning (Preskill & Boyle 2008).

Measurement of M&E capacity-strengthening initiatives is critical for ensuring that state and non-state actors can effectively meet the increasing demand for monitoring and evaluation (Carman & Fredericks 2008). Also, the measurement of M&E capacity-strengthening interventions is crucial because M&E capacity ensures that individuals, institutions and countries can regularly document the implementation of their programmes and can interpret their results, to understand and strengthen programme implementation, improve programme outcomes, and meet the accountability requirements of funders, governments and citizens, as well as accrediting bodies (Taylor-Ritzler et al. 2013).

Current trends in measuring monitoring and evaluation capacity strengthening

The available literature appears to be indicating that M&E capacity-strengthening initiatives in Africa have taken the form of short-term endeavours, including training offered to individuals, technical assistance provided to selected ministries and study visits among grantees, funded mainly by international development agencies (Engela & Ajam 2010; Mackay 2009; Tarsilla 2014). The focus of which has been on ensuring the success of and leveraging the impact of their respective programmes in various African countries (Basheka & Byamugisha 2015). Consequently, these M&E capacity-strengthening initiatives have not resulted in the promotion of a stronger M&E culture within systems but have instead resulted in modest improvement of the technical M&E capacity among a few local staff members working for the same international development partners who fund such initiatives (Morkel & Mangwiro 2019). The consequence of this is that these initiatives promote the smooth implementation of international aid processes, at the expense of context-specific learning and ownership of the evaluation function, which then hinders development effectiveness (Tarsilla 2014).

Literature indicates that while there is a growing demand for M&E capacity to meet the demand for evidence to support policy decisions, guide resource allocation, demonstrate development results and enhance overall performance; the impact of M&E capacity building or development (i.e. capacity strengthening) initiatives has yet to be measured, particularly in the African context (Mapitsa & Khumalo 2018; Morkel & Ramasobama 2017). One of the key reasons for the lack of measurement is that there is no consensus on the definition of M&E capacity, which is referred to as ‘conceptual ambiguity’ around M&E capacity, which therefore results in difficulty in measuring the effects of M&E capacity-strengthening initiatives (Morkel & Ramasobama 2017). Significant amounts of resources have been utilised to improve African countries’ capacity to monitor and evaluate their performance and the impact of their development programmes and policies; however, the literature suggests that most M&E capacity-strengthening interventions have not yielded the expected results (Mackay 2007; Tarsilla 2014). According to Tarsilla (2014), those efforts that do exist have been focused on measuring the short-term effects of M&E capacity-strengthening initiatives, while the long-term effects have been largely neglected. Furthermore, the literature indicates that measurement of the effects of M&E capacity-strengthening initiatives is equally complex, as it can take many forms and may occur at an individual, organisational or system level (Morkel & Ramasobama 2017).

Methods

A qualitative research approach was adopted for this study, and it enabled us to explore and examine the various M&E capacity-strengthening approaches and how they are measured from the perspective of those who are funding and implementing such initiatives. Semi-structured interviews were used to gain an in-depth understanding of capacity-strengthening approaches and their impacts and to draw out information on how the capacity-strengthening activities are measured. Participants for the semistructured interviews were drawn from the following countries: Botswana, Ethiopia, Ghana, Kenya, Rwanda, Tanzania, Uganda and Zambia. These countries were selected because they are key targets for M&E capacity strengthening by the institution to which the researchers are attached to. The interviews were conducted either telephonically or in-person with purposively selected participants drawn from entities such as national government M&E departments, international non-governmental or multilateral institutions and local non-governmental institutions. All participants are senior level officials within their organisations and are directly involved in M&E capacity strengthening, either at country or regional level.

The criteria for selecting the participants for the in-depth enquiry were involvement and experience in M&E capacity development or building and/or presently working in the monitoring and evaluation field either in the state sector or in the non-state sector. The study targeted to interview 20 respondents from selected countries; however, saturation was reached at fifteen interviews. The categories of the study sample are shown in Table 1.

TABLE 1: In-depth inquiry participants across selected eight countries.
Data analysis

Data analysis was framed around the study objectives and research design. Qualitative data from the interviews were transcribed and analysed using content analysis. The researchers initially coded the data and grouped the codes into categories that were grouped into descriptive themes. The coding facilitated the research team to make sense and meaning of the content and, more importantly, to sort data by thematic areas linked to the study objectives. Direct quotes were not used in this article because the aim was to portray the overall story that the information tells or the general perspective it provides.

Limitations

The major limitation of the study is that the findings cannot be generalised to the study population or community because of the methodology and research design adopted which does not allow the findings to be generalised across the countries. Although the study findings are not generalisable across the selected countries, the researchers engaged an external researcher with expertise in M&E in Africa to conduct dependability analysis through an inquiry audit to ensure that the findings are consistent and could be repeated. Despite the inquiry audit, readers have to take caution when applying the findings to the overall countries as the study might not have been exhaustive.

Ethical considerations

Ethical clearance to conduct this study was obtained from the University of the Witwatersrand Human Research Ethics Committee (reference number: H19/11/32).

Results

Overview of organisations implementing capacity strengthening

There are various institutions that are involved in M&E capacity-strengthening activities in the selected countries. These institutions can be classified under the following categories: state institutions, higher institutions of education and learning, multilateral organisations, non-governmental organisations, M&E professional associations and individual consultants. Multilateral organisations such as the World Bank, United Nations agencies (United Nations Children’s Fund [UNICEF], United Nations Development Programme [UNDP], United Nations Population Fund [UNFPA]) and the African Development Bank (AFDB) were identified as involved in capacity-strengthening activities across several African countries; however, their approaches vary from country to country. International non-governmental organisations and government agencies such as United States Agency for International Development (USAID), Department for International Development (DFID), Swedish International Development Cooperation Agency (SIDA), Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ), International Initiative for Impact Evaluation (3ie), African Development Bank and Centre for Learning on Evaluation and Results in Anglophone Africa (CLEAR-AA) are also involved in either funding or implementing M&E capacity-strengthening initiatives across the eight African countries. There are also other sector-specific organisations that focus on strengthening the M&E capacities of their staff only to meet their M&E accountability obligations and that the state sector benefits little from these efforts. Although the activities are limited to their staff, they contribute to the broad ecosystem of M&E capacity strengthening.

State departments in Botswana, Kenya, Tanzania and Uganda are also involved in M&E capacity strengthening, this is in addition to their country-level M&E coordination roles. Uganda’s Prime Minister’s office, Botswana’s National Strategy Office, Tanzania’s Public Service Management and Good Governance Office and Kenya’s M&E Department are actively involved in M&E capacity strengthening. Their involvement should be viewed with the following issues: First, multilateral institutions and other organisations play a significant role in these countries through providing funding that enables these state departments to execute their M&E capacity-strengthening activities. In return, the multilateral institutions drive the M&E capacity-strengthening agenda. Second, state sector capacity-strengthening activities are centred around the capacities of government institutions and have limited reach in organisations outside the state system. The non-state sector only benefits when the trained staff leave and join the non-state sector and also through the state creating an enabling environment for the practice of M&E.

Higher institutions of education and learning were also indicated as providing M&E capacity strengthening across all the eight countries covered in this study. These institutions offer short M&E courses that are targeted at individuals. A review of these institutions shows that M&E course are mainly pegged at postgraduate level hence no direct access for those who want to study M&E from undergraduate level.

M&E professional associations (Voluntary Organizations for Professional Evaluation [VOPEs]) are also indicated as organisations that are active in M&E capacity strengthening in all selected countries except Botswana. Participants highlighted that although these associations are not yet fully developed in some cases, they play a significant role in M&E capacity strengthening.

Country system-level capacity strengthening

There is a perception by participants that system or country-level activities such as the development of M&E policies and frameworks, creating an enabling environment and developing institutional M&E systems do not constitute M&E capacity development or building. The participants’ definition of capacity is limited to those and focuses on activities that strengthen individual skills and knowledge. At the country M&E system level, the focus is on developing policies, frameworks and M&E systems specifically for state institutions and is dominated by multilateral institutions. These approaches are being implemented in collaboration with other international organisations. The various capacity-strengthening activities appear to concentrate on the formulation of M&E policies and guidelines to guide the institutionalisation of these components in the state sector rather than on the implementation of M&E guidelines and policies within the state sector.

The institutionalisation of monitoring and evaluation (institutional level)

Capacity-strengthening activities at the institutional or organisational level are classified into state sector and non-state sector activities. State sector activities in Botswana, Kenya and Tanzania aim to institutionalise M&E within state departments through the development and capacitation of M&E units within ministries and departments. In Botswana, concerted efforts are being made to set up M&E units in government departments whilst in Uganda a few of the ministries have appointed M&E officers but others still lack M&E structures because of resource constraints. In the non-state sector, the situation is somewhat different with various approaches having been implemented to strengthen M&E structures, processes and capacitation of individuals. A participant summed up this scenario by saying:

‘[T]he non-governmental organisations which receive funding from international organisations are required to follow stringent accountability requirements which a functional organisational M&E system can provide and they, therefore, have an obligation to build their capacity.’ (Non-state sector, Male, Regional organisation)

However, despite having this capacity, the benefits of the system do not spill over to the stateside because of the lack of platforms and interlinkages to do so.

Approaches to developing individual skills and abilities

At an individual level, several organisations are playing a critical role in strengthening individual skills and abilities. It is also important to note that because of the perception that M&E capacity is mainly individual, participants regarded it as the main activity. The focus is on strengthening M&E individual activities so that they are able to deliver on development projects and improve organisational reporting, mainly for accountability purposes. The approaches used at the individual level only target M&E practitioners and to a lesser extent decision makers. These activities include training individuals (M&E practitioners) on specific skills, for example, impact evaluation, data analysis and report writing; on-the-job M&E training – internships, mentoring and coaching; higher training institutions offering training (sector-specific M&E courses) and pre-conference training workshops to participants.

A common issue that was raised by the study participants is that individual M&E capacity strengthening lacks coordination and several entities work in silos. The implication is that effectiveness of such approaches becomes limited and difficult to measure.

Capacity-strengthening enabling factors
Strides to institutionalise monitoring and evaluation

There are several factors that enable capacity-strengthening in the selected countries. These factors include state-driven interventions that seek to institutionalise M&E systems and fund it from state budgets. The result for this is that countries have formalised M&E units within state departments and also developed M&E policies that are linked to the national development agendas. This aids the development of an enabling environment that furthers the M&E capacities within the specific countries.

Availability of international funding

The role of development funders or donors was highlighted as enabling capacity strengthening. Across the eight countries, it was highlighted that funding for development programmes is now seldom granted without the imposition of stringent M&E requirements and the availability of funding to capacitate staff of the organisations receiving the funding. This has played a significant role in building the M&E human capital of the various countries and furthering the development of M&E institutional capacity in both state and non-state institutions.

Growing demand for monitoring and evaluation

Growing demand for M&E products within state institutions was cited as one of the factors that is spurring M&E capacity strengthening. Although participants indicated that there are challenges in situations where the importance of M&E is undervalued and there is a poor appreciation and awareness of M&E by politicians, demand for M&E products was increasing, thereby opening avenues for capacity strengthening within the state. Participants highlighted that M&E champions within state institutions have played a significant role in creating this demand. Champions at this level of authority have a valuable role to play in the integration of M&E across the public policy cycle at the sectoral and national level.

State and non-state partnerships

As highlighted previously, there are several partnerships between state institutions and non-state institutions that aim to strengthen M&E capacities in the selected countries. Participants reported that partnerships between stakeholders such as VOPEs, government departments and donors are common in various countries. These partnerships have led to the execution of various capacity-strengthening activities that include developing M&E policies and frameworks at the national level, M&E institutionalisation and development of individual skills and abilities. Participants regarded these partnerships as important in strengthening M&E capacities; however, the partnerships do not extend to the practice of M&E. The non-state and state sectors operate in silos with the former having developed efficient M&E capacities, while the latter is poorly funded and struggles with limited human resource capacity.

Barriers to effective monitoring and evaluation systems capacity strengthening
Scarce resources

Limited funding for M&E capacity strengthening was regarded as the main obstacle to capacity strengthening. The state sector allocates limited funding for M&E capacity strengthening and implementation of M&E policies. State institutions depend too heavily on international donors for M&E capacity strengthening. Within the non-state sector, the situation is completely different because of funding guarantees for M&E system capacity strengthening and the setting up of M&E systems to improve accountability and performance monitoring. At the individual level, there are scarce resources available for the academic and professional training for individuals not attached to institutions.

Limited culture of evaluation

The perennial problem of lack of a culture of evaluation was also cited as a barrier to M&E capacity strengthening. Two reasons were preferred to support this perspective; first, M&E within the public sector is regarded as a policing tool and a type of audit; hence, M&E practitioners are regarded as performing ‘police work’ or checking what went wrong instead of using it as an opportunity to learn. Second, within the non-state sector, M&E is also regarded mainly as an accountability tool rather than a tool that can inform decision-making. Participants argued that although there is a growing demand for M&E, development outcomes might not improve if M&E is regarded as a policing and accountability tool only.

Limited understanding of the role of monitoring and evaluation by critical stakeholders

Participants also highlighted that there is a lack of appreciation and awareness for M&E especially among the state bureaucrats, political actors and some of the funders, thereby limiting the M&E capacity-strengthening activities. Other study participants argued that although there is a growing demand of M&E products, there is a lack of understanding of how to use the products there by affecting evidence-based decision-making. This has reduced the status of M&E and has indirectly affected the availability and drive of capacity-strengthening resources.

Mistrust in sharing monitoring and evaluation products

The development of M&E systems in the selected countries has been spearheaded by the international donor community. Although the international community and the state sectors are addressing the same development problems, there is a pervasive nature of relationship on how they share M&E products leading to the fragmentation and working in silos. The non-state sector is more focused on building systems that are primed to report to their funders and not their state sector counterparts. This limited sharing of information between non-state sector and state sector in more inclusive societies, such as Ghana, is largely because of the lack of coordination mechanisms for centralising government and civil society-generated data. In other countries that are less inclusive, such as Uganda, there tends to be mistrust regarding how the data will be used.

Monitoring and evaluation curricula and training barriers

The non-availability of M&E courses at the undergraduate level is cited as being directly responsible for the human capacity shortages across African countries and the poor development and adaptation of M&E as a discipline in the African context. While professional courses, workshops and postgraduate academic courses are being offered across Africa, these are frequently beyond the reach of the very individuals and institutions in critical need of training. Organisations and government departments that lack resources have very limited access to these professional courses while non-state entities that have the necessary funds can afford to capacitate their institutions by sending individuals for professional development courses. Also, because of the disparity in remuneration between state and non-state organisations (INGOs) in Africa, qualified M&E individuals tend to migrate to the non-state sector. State institutions effectively become a training ground for the non-state sector as the most capable and qualified M&E human capital moves out of the state.

Unknown monitoring and evaluation capacity needs

Across the eight countries, the M&E capacities of individuals, country and institutions are not known. This has been identified as a barrier to capacity strengthening. The participant perception is that M&E capacity needs and gaps are not known and funding organisation focus on capacity areas they have an interest in not necessarily the one with the most need. The implication of this is that M&E capacity-strengthening design, approaches and implementation is driven and under control of international funders. Participants argued that it is prudent for countries to assess their M&E capacity needs and prioritise the meagre resources towards areas with the greatest need.

How is monitoring and evaluation system capacity strengthening measured?

Given the resources that are invested in M&E capacity strengthening in the selected countries and Africa in general, it is critical to measure the effectiveness of these activities. There is a consensus among the study participants that capacity initiatives results are realised in the medium to long term. The implication is that the impact of capacity-strengthening initiatives is noticed after a much longer period. The fact that the major players, both funders and implementers of capacity-strengthening activities are from outside the country means that their interventions are targeted at a specific area or level and are limited in time. This also means that the resources for measuring the effectiveness of M&E capacity strengthening are also available for a short period; hence, it is often difficult to measure the long-term impacts of the capacity-strengthening activities.

The second issue raised in this study is that capacity-strengthening activities are viewed as a very process-oriented and form a small component of the whole programme cycle. There is a perception that these initiatives are not worth reporting or evaluating at the end of the cycle. The implication is that not much thought is put in the design of the capacity-strengthening activities making such activities ad hoc. In addition, capacity-strengthening activities do not receive adequate funding and support by decision-makers from the state leading to limited resources for evaluating the activities. One participant from one of the state departments summarised the implication of not measuring M&E capacity development by saying:

‘Despite significant investment being made in the African context to develop evaluation capacity, both state and non-state institutions decry the lack of evaluation skills within their organisations and the sector still depends on foreign evaluators.’ (State department, Male, Public sector)

The participants assigned the blame for the lack of these skills for not evaluating the design, processes and outcomes of M&E capacity-strengthening activities. The argument is that if those funding and implementing M&E system capacity development were insisting on measuring the outcomes then it will lead to well-thought-out, informed solutions to addressing capacity strengthening.

What has been measured and how?

To explore how measurement of capacity-strengthening is occurring, participants were asked to list some of the approaches that are being used. They listed the following commonly used indicators: pre and post training scores, number of individuals trained, end of module assessments, number of courses held and number of graduates. It is pertinent to note that participants listed mainly outputs indicators as a form of measurement, and these indicators were only revealed after pressing the participants, which shows that measuring capacity-strengthening approaches is not well thought out. Furthermore, the participants were asked about intermediate outcomes, and they identified two indicators that are the number of civil society organisations with the ability to produce quality data in their programmes and ability to perform data quality assurance. Only four participants listed wider outcome indicators around changes in behaviour or performance that are related to the broader goal of capacity strengthening.

To remedy the lack of measurement and the poorly defined measurement metrics, participants proposed the following indicators to measure M&E capacity strengthening: quality of reporting, quality of evaluations, quality of data produced, timeliness of reporting – is reporting aligned to the decision-making process, evidence use in decision-making by both state and non-state actors, use of performance data for programme learning and adaptation at the institutional level and uptake of M&E in the decision-making process.

The results have shown that issues to do with the measurement of M&E capacity-strengthening activities are not divorced from the broad issues of capacity strengthening. The fact that the majority of capacity-strengthening activities are focusing at the individual level means that performance-monitoring indicators are more dominant than the system-level outcomes. Participants highlighted that compounding factors make it difficult to measure outcomes such as problems with disjuncture in the design of capacity-strengthening activities, an unsystematic approach and the disjuncture between what the training curriculum offers and the M&E sector skills requirements. Given the observations presented, it can be argued that for effective measurement of the outcomes of M&E capacity-strengthening activities, a systematic approach to capacity strengthening is needed, and the design should address known capacity needs within institutions and at the system level. Such an approach addresses the non-availability of resources for measuring outcomes and refocuses capacity strengthening to national needs.

Discussion

This study has shown that M&E capacity strengthening in the selected countries is ad hoc, indiscriminate, haphazard and mainly focuses on developing individual skills and abilities. These issues are also reflected in how these capacity-strengthening activities are designed and measured. The non-availability of well-planned M&E capacity-strengthening initiatives and measurement of these activities limits learning and identification of best practices in M&E capacity strengthening. Furthermore, it is important to note that for these initiatives to achieve positive outcomes, the design and implementation of approaches must address the gaps and take cognisance of the context in which capacity-strengthening activities are to be implemented. Therefore, it can be argued that using M&E capacity-strengthening initiatives designs from other contexts should be done cautiously and there is need to interrogate their sustainability and suitability to the local context. Context is also pertinent because it informs the capacity needed, the value attached to M&E and the extent to which it will influence the decision-making process by both state and non-state entities.

The ad hoc, indiscriminate, haphazard and piecemeal nature of M&E system capacity strengthening is as a result of limited M&E situational or needs assessment. As an underlying principle, M&E capacity strengthening must be preceded by a situational and needs assessment of the intended beneficiaries, it must be demand driven and must respond to specific needs. Without a situational analysis or needs assessment, M&E system capacity strengthening will remain supply-side driven. Good practice, in this case, demands that M&E capacity strengthening be driven from within and although there is a greater likelihood of success if the design of the interventions is based on demand, attention to the incentives behind the demand is essential.

Furthermore, this study has shown that M&E capacity-strengthening initiatives that go beyond individual skills and abilities to include institutional and system-level capacity are likely to yield better outcomes. There is a need to dispel the commonly held notion that focusing on individual skills-sets and abilities automatically lead to significant improvements in institutional and system-level capacity. The design for M&E system capacity-strengthening initiatives must do more than improving individual competencies and improve institutional and system-level capacities.

The findings also revealed several key issues that are important to strengthening M&E capacity in the African context and are aligned to the available literature. Firstly, throughout all eight countries, capacity needs assessments and capacity-strengthening plans are lacking. Needs assessment has been identified by Watson (2006) and Heider (2011) as the first step to understanding the gaps in capacities and the various components of the system. In addition, needs assessments will lead to the development of contextualised strategies to address the gaps. The failure to define country capacity needs has led to the narrow perception that capacity is limited to the micro-level and a distinct lack of understanding of the inherent interconnectedness of all levels of capacity.

Secondly, the issue is that capacity strengthening is not aligned with the current contextual factors such as culture, socio-economic, political and governance systems. The most preferred approach is one size fits all – this approach fails to acknowledge that M&E capacity determinants such as culture, the value of M&E and the general environment of M&E practice are endogenous and context specific (Carter 2013). It can be inferred that such a scenario results in a mismatch between the supply and demand aspects of capacity.

Thirdly, this study revealed that current M&E capacity-strengthening initiatives do not reiterate the key components that are crucial for the comprehensive development of M&E systems. Capacity strengthening at a macro level (countries and institutions) should be creating an enabling environment that enables and values the practice of M&E as stated by Mackay (2007). This is achieved through assisting countries in developing coherent policies, culture, structures, strategies and inducing behaviour change, a process that will bring new values at both micro and macro levels of capacity. The creation of an enabling environment within the state sector is a mammoth task that is further hampered by a lack of human capacity, resources, absence of policies and frameworks and weaker demand for M&E compared with the non-state sector. This has a direct implication on state programmes’ effectiveness, accountability and impedes the development of a culture of evaluation.

Fourthly, the findings of this study show that limited resources (material, human and financial) are available across all levels (individual, institutional and state level) to develop M&E capacities. At the state level, the sustainability of M&E capacities in government MDAs is affected by the concerted dependence on non-state funders. This dependence is also problematic because funders have the proclivity to dictate the capacity-strengthening goals and approaches to be espoused, which means that resources are not utilised where they are needed most. While international organisations provide sufficient funding for non-state sector institutions to build M&E system capacity and establish M&E systems to improve accountability and performance monitoring. Yet, no funding is given for the measurement of the efficiency, effectiveness and impact of these M&E capacity-strengthening approaches. Similar findings were reported in a study done by Watson (2006), which found that there is little consensus on the best way to identify and measure M&E capacity development. At the individual level, access to academic and professional training by private individuals is hindered by the scarcity of resources, particularly in impoverished African countries.

Fifthly, the findings demonstrate that the measurement of M&E capacity-strengthening activities is not clearly defined, amorphous and ambiguous. In addition, the only stages where the M&E capacity strengthening is measured is at the input and output stages of the micro-level spectrum. The lack of monitoring tools for assessing the efficiency and effectiveness of capacity-strengthening activities prevents implementers from determine what is working, not working, for whom and under what circumstances. Also, the lack of evaluation of M&E capacity-strengthening activities means that the relationship between these activities and M&E capacity outcomes is not defined.

Conclusions and lessons learnt

The significance of strengthening M&E system capacity in Anglophone Africa has been strongly supported by the findings from this study, considering the critical impact that effective M&E systems have in enabling countries to reach their development goals. Additional research is required to gain comprehensive understanding of the various approaches to country M&E system capacity strengthening, and how the design and training curricula can be adapted to specific country needs and contexts. Despite the methodological confines of this study, four key lessons were identified that are important to individuals and organisations with an interest in or who are engaged in M&E capacity strengthening in Africa.

Firstly, funding should be allocated to M&E capacity needs assessments as these are crucial at the country level to ascertain the kind of capacity that is available and its effect on the M&E practice in the country. In addition, a capacity development plan that is tailored to each country context is required and should indicate the required capacity improvements or provide recommendations on new kinds of capacity that might be needed. Also, linked to this first lesson is that measurement tools are needed for examining merit, effectiveness and M&E capacity-strengthening approaches. Implementing the capacity development plan will require a robust knowledge management system accessible to both state and non-state organisations to source information that indicates what is working, what is not working, for whom and under what circumstances.

Secondly, funders or implementers of capacity-strengthening activities should prioritise the entire M&E system and strengthen all levels of capacity, including individual, institutional and country levels. The emphasis must transcend the monitoring of micro-level technical skills and abilities, however, must also incorporate the strengthening of macro-level capacity and evaluation skills.

Thirdly, it is vital to note that countries are heterogeneous; thus, M&E capacity is mainly reliant upon on the contextual factors and nuances prevalent in each organisation and country. The people or institutions engaged in designing capacity-strengthening approaches in M&E should create approaches that are context-specific, which are of a satisfactory standard to the recipients of these initiatives and that enable easy adaptation, measurement and sustainability of the gained capacity.

Finally, African governments and international development partners implement initiatives addressing the same development challenges; however, there are vast differences in the M&E capacities of these institutions. Synergies between state and non-state sectors are required to remove the M&E capacity silos that currently exist and to establish opportunities for measuring the effectiveness of approaches, mutual learning and sharing to ascertain the kind of strategy that works well in their particular country. The government should lead the development of these synergies.

Acknowledgements

The authors acknowledge CLEAR-AA staff who contributed to conducting the strengthening and measuring M&E capacity in selected African programmes research study, collected the data and wrote the original research report. Special thank you to all sampled staff of organisations responsible for M&E system capacity strengthening in Botswana, Ethiopia, Ghana, Kenya, Rwanda, Tanzania, Uganda and Zambia for participating in the research study.

Competing interests

The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.

Authors’ contributions

S.M. contributed to the original research study and writing the original research report, the conceptualisation of the journal article based on the study and the development of this article. T.F. contributed to reviewing and editing the original research report and writing of the journal article.

Funding information

This research received funding from CLEAR-AA.

Data availability

The data that support the findings of this study can by made available by the corresponding author, T.F., upon reasonable request.

Disclaimer

The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of any affiliated agency of the authors.

References

Babu, S.C., 2018, Monitoring and evaluation for capacity strengthening, vol. 2, International Food Policy Research Institute, Washington, DC.

Basheka, B.C. & Byamugisha, A.K., 2015, ‘The state of monitoring and evaluation (M&E) as a discipline in Africa: From infancy to adulthood?’, African Journal of Public Affairs 8(3), 75–95, viewed n.d., from https://hdl.handle.net/2263/58169

Carman, J.G. & Fredericks, K.A., 2008, ‘Nonprofits and evaluation: Empirical evidence from the field’, New Directions for Evaluation 2008(119), 51–71. https://doi.org/10.1002/ev.268

Carter, B., 2013, Evaluation capacity development, Helpdesk Research Report, GSDRC, viewed n.d., from https://gsdrc.org/docs/open/hdq996.pdf

Engela, R. & Ajam, T., 2010, Implementing a government-wide monitoring and evaluation system in South Africa, The World Bank, Washington, DC.

Farrell, G.M., 2009, Results-based monitoring and evaluation at the Commonwealth of Learning, Commonwealth of Learning (COL), Vancouver.

Görgens, M. & Kusek, J.Z., 2010, Making monitoring and evaluation systems work: A capacity development tool kit, The World Bank, Washington, DC.

Heider, C., 2011, ‘A conceptual framework for developing evaluation capacities: Building on good practice’, in R.C. Rist, M. Boily & F. Martin (eds.), Influencing Change: Building Evaluation Capacity to Strengthen Governance, The World Bank, Washington, DC.

Kemerer, V. & Salentine, S., 2014, A case study to measure national HIV monitoring and evaluation system strengthening: Côte d’Ivoire, 1–50, viewed n.d., from https://www.measureevaluation.org/resources/publications/sr-14-102.html

Lopez-Acevedo, G., Krause, P. & Mackay, K., 2012, Building better policies: The nuts and bolts of monitoring and evaluation systems, World Bank Publications, Washington, DC.

Mackay, K., 2009, ‘Building monitoring and evaluation systems to improve government performance’, in M. Segone (ed.), Country-led monitoring and evaluation systems: Better evidence, better policies, better development results, pp. 169–174, UNICEF, New York.

Mackay, K.R., 2007, How to build M&E systems to support better government, The World Bank, Washington, DC.

Mapitsa, C.B. & Khumalo, L., 2018, ‘Diagnosing monitoring and evaluation capacity in Africa’, African Evaluation Journal 6(1), 1–10. https://doi.org/10.4102/aej.v6i1.255

Morkel, C. & Mangwiro, N., 2019, ‘Implications of evaluation trends for capacity development’, in C. Blaser-Mapitsa, P. Tirivanhu & N. Pophiwa (eds.), Evaluation landscape in Africa: Context, methods and capacity, pp. 184–200, African Sun Media, Stellenbosch.

Morkel, C. & Ramasobama, M., 2017, ‘Measuring the effect of evaluation capacity building initiatives in Africa: A review’, African Evaluation Journal 5(1), a187. https://doi.org/10.18820/9781928480198/08

Nuyens, Y., 2005, No development without research, a challenge for research capacity strengtherning forum 10, October 2006, Report: Combating Diseases And Promoting Health, Global Forum For Health Research, Geneva.

Preskill, H. & Boyle, S., 2008, ‘A multidisciplinary model of evaluation capacity building’, American Journal of Evaluation 29(4), 443–459. https://doi.org/10.1177/1098214008324182

Struyk, R.J., Damon, M. & Haddaway, S.R., 2011, ‘Evaluating capacity building for policy research organizations’, American Journal of Evaluation 32(1), 50–69. https://doi.org/10.1177/1098214010378355

Tarsilla, M., 2014, ‘Evaluation capacity development in Africa: Current landscape of international partners’ initiatives, lessons learned and the way forward’, African Evaluation Journal 2(1), 13. https://doi.org/10.4102/aej.v2i1.89

Taylor-Ritzler, T., Suarez-Balcazar, Y., Garcia-Iriarte, E., Henry, D.B. & Balcazar, F.E., 2013, ‘Understanding and measuring evaluation capacity: A model and instrument validation study’, American Journal of Evaluation 34(2), 190–206. https://doi.org/10.1177/1098214012471421

Watson, D., 2006, ‘Monitoring and evaluation of capacity and capacity development’, in ECDPM (ed.), Capacity, change and performance, pp. 1–32, ECDPM, The Centre for Africa-Europe relations, Maastricht.

Weiss, C.H., 1998, ‘Have we learned anything new about the use of evaluation?’, American Journal of Evaluation 19(1), 21–33. https://doi.org/10.1177/109821409801900103

Woodhill, J., 2019, ‘M&E as Learning: Rethinking the Dominant Paradigm’, in Monitoring and evaluation of soil conservation and watershed development projects, pp. 83–108, USAS, CRC Press, Florida.


 

Crossref Citations

1. Effect of Monitoring and Evaluation Practices on the Performance of Education-Funded Projects in Rwanda: A Case of the Building Learning Foundation (BLF) Project in Gasabo District
Jean Felix Rugango Mugisha, Kengere A. Osiemo
African Quarterly Social Science Review  vol: 1  issue: 3  first page: 67  year: 2024  
doi: 10.51867/AQSSR.1.3.6