Since the early 2000s, African evaluation experts, scholars and voluntary organisations for professional evaluation have worked towards developing evaluation approaches based on African indigenous philosophies (Uwizeyimana 2020). This type of approach to development evaluation has been dubbed ‘Africa-rooted evaluation’ or ‘Made in Africa Evaluation’ (MAE). The MAE agenda was made central to the aspirations of the African Evaluation Association (AfrEA) at its inception in 1999, culminating in MAE as a pillar of its 2018–2021 strategy (IDEV 2019). Since the early 2000s, the concept of MAE has gained traction, mainly through the agenda of decolonising knowledge and promoting Africa’s epistemic identity. Chilisa and Mertens (2021:245) define MAE as ‘an approach that embraces African resistance to blind borrowing of Western values and standards to evaluate programs in Africa’. In broad terms, the approach provides impetus for African evaluation experts and scholars to deliberately adapt evaluation tools, instruments, strategies and models towards developing evaluation practice, theory and methodologies grounded in local cultures, indigenous knowledge systems and African philosophies. However, although the approach is rooted in African worldviews, it embraces other knowledge systems (Chilisa 2020). Therefore, a fundamental issue is in the grounding of development evaluations within African epistemologies, African indigenous knowledge systems and African-rooted evaluation methodologies.

The MAE agenda is in line with the contemporary discourse on decolonising knowledge for epistemic freedom in Africa. In this context, Chilisa et al. (2016) assert that decolonisation of evaluation entails restructuring of power relations in the current global architecture of evaluation knowledge generation. This will allow African evaluators to play a meaningful role in the conceptualisation of what is evaluated, when it is evaluated, by whom and with what methodologies. In this regard, emphasis is placed on the need for appreciating context, culture, history and beliefs to shape the nature of evaluations, in recognition of diverse, dynamic and complex African socio-economic development realities. Such an approach will enhance the relevance and applicability of knowledge and evidence from evaluations. In fact, Mbava (2017:141) argues that current recommendations from most evaluations do not pay sufficient attention to variations in local contexts where programmes are implemented. Additionally, Cloete (2016:1) argues that currently, evaluation practice in Africa is based on external values and contexts. It is highly donor-driven, with accountability mechanisms skewed towards aid recipients rather than an accountability and learning mechanism for both aid recipients and funders.

There is consensus by proponents of MAE on the fact that current evaluation principles, assumptions and practices which are based on Eurocentric epistemologies are frequently found to be unsuitable in the African context (Chilisa & Malunga 2012; Uwizeyimana 2020). According to Uwizeyimana (2020), these current approaches focus more on efficiency, effectiveness and economic development, instead of addressing issues of communal well-being, empathy and indigenous cultural values and traditions – which are more relevant to the African context. In addition, it is believed by these proponents that African rooted evaluation will bring much deeper understanding on the macro–micro disconnects, attribution, ethics and values, and power relations. In this regard, an MAE approach will likely contribute towards achieving some of the sustainable development goals (SDGs) through their inherent capacity to raise issues about culture, ethics, values and paradigmatic stances in development work (Chilisa & Mertens 2021). In this context, Chilisa and Malunga (2012) suggest two critical transformations in the current evaluation design and practices that will facilitate the transition towards MAE. These are adaptation and domestication of the Western evaluation theory and practices to ensure relevance to the African context and development of relational evaluation paradigms based on the African indigenous philosophy of ubuntu.¹ Such a transformation, through the MAE approach, will likely improve the relevance of evaluation to African contexts and enhance effective utilisation of evaluation results for social change.

Although efforts to make evaluation culturally relevant have become central to evaluation discourses globally, global attempts at culturally responsive practice have not succeeded in incorporating African voices (Chilisa et al. 2016). Evaluation scholars, academics and practitioners must overcome several hurdles if the MAE is to have meaningful success. According to Uwizeyimana (2020), some of the critical issues include the need by indigenous African evaluation and nonindigenous African scholars and experts to agree on the African values, practices and traditions on which the MAE approach should be based. Additionally, the indigenous African philosophy of ubuntu has its own downsides which have serious implications for MAE. It is therefore important for some of these shortcomings to be made explicit before making ubuntu as the foundation of MAE. Conceptually, key challenges include the proposal for adopting and indigenising the existing Eurocentric evaluation theories and practices without fully exploring and having consensus on the key tenets of MAE (which include mainstreaming African epistemologies in evaluations).

Much of the discourse on MAE to date has focused on the conceptual aspects (IDEV 2019). Several grey areas remain on how MAE can be put into practice. Omosa et al. (2021), for example, advocate for reviewing the AfrEA guidelines and considering evolving definitions of MAE. Although a review of literature points to publications that outline some components for rolling out MAE in practice (e.g. Chilisa, Major & Khudu-Petersen 2017; Cram, Tibbetts & LaFrance 2018), there is a need for a more systemic way of looking at the practice implications of MAE. This article aims at filling this research gap through exploring the praxis implications of MAE. The article addresses the following research question: what are the critical components for MAE development evaluation practice? An exploratory research design is adopted, guided by theoretical constructs from critical systems heuristics (CSH). The assessment is guided by existing evaluation frameworks, practice guidelines (including the African Evaluation Guidelines – Standards and Norms) and theoretical and methodological guidelines. Data collection was based on secondary literature reviews and expert and experiential knowledge regarding development evaluation practice. It is envisaged that the research findings will contribute towards strengthening the MAE agenda.

Development evaluations aim at providing results, evidence and knowledge that informs policies, programmes and projects aimed at inducing change in social systems. In this regard, evaluations are commissioned as interventions to transform society or social systems. This invariably calls for a conceptual framework that evaluates design and implementation or results of interventions for transforming social systems. This research therefore adopts a conceptual framework based on CSH, which provides a framework for a critical reflective process on the design of social interventions. Critical systems heuristics has foundations in the seminal work of West Churchman in the 1960s focusing on transformation of social systems through interventions (Midgley 2000). It offers an approach for socially rationalising plans, interventions and development practice (Luckett 2006) and interrogation of social systems design or boundaries and assessment of their social implications (Flood & Jackson 1991). Critical systems heuristics is a strand of critical systems thinking, within the field of applied systems thinking, concerned with the normative core of professional practice and how professionals can (and should) reflect on, critique and justify the values and assumptions that influence their work (Ulrich 2012). The central argument advanced here is that CSH can support reflections on the design of approaches to development evaluation practice through being critically reflective. Being critically reflective means identifying, questioning and justifying the values and assumptions that influence and should influence evaluation practice.

Critical systems heuristics is concerned with discursive acts, with decisions made by multiple parties with varying goals, epistemic frameworks and ways of describing the world and the systems being designed. Rather than seeking to classify the component elements of an assumed system or provide a model of their relations, CSH focuses on the reflexive consideration of a designed system’s purpose or goals, and how these are justified by a ‘reference system’ of assumptions and judgements (McCord & Becker 2019). Therefore, in this research, CSH is utilised to assess how evaluation practice within the context of MAE can be designed as a social intervention. Critical systems heuristics as a framework for reflective professional practice is organised around the central tool of boundary critique (Ulrich & Reynolds 2010). Boundaries are what we socially construct in designing and evaluating any human activity system of interest. Boundary judgements determine which empirical observations and value considerations count as relevant and which others are left out or are considered less important. Because they condition both ‘facts’ and ‘values’, boundary judgements play an essential role when it comes to assessing the meaning and merits of a claim.

The primary boundary of any human activity system is defined by the ‘purpose’ that it is expected to serve. Therefore, CSH is utilised to surface, elaborate and critically consider boundary judgements, that is, the ways in which people or groups decide what is relevant to the system of interest (any situation of concern). The CSH questions can be applied to any purposeful system of interest, that is, any area or situation of concern that might be associated with human purpose, whether it is of individual or collective concern, that is, evaluating the actual purposes and implications of purposeful activity with relevant stakeholder groups (Reynolds 2007).

In this research, the assessment of components that define the practice of MAE is premised on the assertion that the practice of MAE is designed based on defining system boundaries that determine what is regarded as pertinent for driving social transformation. The process of defining system boundaries is judgemental based on an interaction between knowledge, morals, values and beliefs (Romero 2002:45–46). In this regard, assessing the design or conceptualisation of MAE in practice therefore implies critiquing boundary decisions. The process encompasses an assessment of the socio-economic, sociocultural and ethical implications of boundaries as a way of rationalising MAE in practice.

Critical systems heuristics is viewed as ‘critical’ in the sense of unearthing values, world views, assumptions and beliefs for social transformation incorporated in defining boundaries. Ulrich (1983) brought clarity to this assertion through the following quotation:

The approach is ‘heuristic’ in its ability to discover or expose ‘objectivist’ deceptions through critical reflections. In professional practice, heuristic procedures serve to identify and explore relevant problem aspects, assumptions, questions or solution strategies. At best, CSH can seek to reveal and problematise the normative assumptions informing a plan, making clear the contingent nature of these boundaries and making them the subject of deliberation. In the context of the practice of MAE, this entails that the design should not be taken as the only objective designs, but rather it should be reflected on through dialogue to incorporate multistakeholder values and world views. It is ‘systemic’ in the sense of reflecting the lack of a priori comprehensiveness or holism in the design of the practice of the MAE approach in terms of the totality of relevant conditions (ethical, political, cultural, ideological, etc.), under which rational decision-making and action take place (Flood & Jackson 1991; Luckett 2006; Ulrich 1983:21). Critical systems heuristics unearth power dynamics and recognise that power can distort purposeful systems through unquestioned decisions and assumptions.

Ulrich (1983) developed a set of boundary questions to unearth and critique assumptions on system boundaries. These interrogate what the system ‘is’ (normative mode) and what the system ‘ought’ to be (objective mode), and the answers are contrasted. In this research, the normative mode for the MAE is assumed to be informed by the African evaluation principles (AEPs) forming the reference system. The principles make cognisance of African knowledge systems and values; they are very much in sync with the MAE agenda. They reflect the demand for evaluation that is ‘Made in Africa’ – proudly tailored by and for African contexts, needs and knowledge systems, yet informed by international good practice insights, theories and practices. The current evaluation practice is used as the ‘reference’ system, identifying first the ideal purpose of the system of interest being evaluated in the ‘ought’ mode. The AEPs (AfrEA 2021) provide five key principles that provide a framework for evaluation practice conducted in and for Africa. These are further disaggregated into 22 implementation principles outlined in Table 1.

The 12 CSH boundary questions are premised on the assumption that designing of purposeful systems is based on four sources of influence, which are motivation for the design, control and power dynamics, expertise required for the system, and legitimating and collective mobilisation (see Figure 1).

In this research, rather than utilising these questions as templates, the questions were utilised in developing a set of questions deemed appropriate for the assessment of MAE in practice. This is in line with debates on application of CSH (e.g. Flood & Jackson 1991:301; Midgley 2000:225) that argue for flexibility and creativity in application of CSH. The key boundary questions applied in this research based on Ulrich’s four areas of influence are illustrated in Table 2.

As outlined in the preceding sections, a fundamental shift in the promotion of MAE is the need to decolonise evaluation and shift from a Eurocentricity-driven evaluation enterprise towards Afrocentric practice. Development evaluation has inherent power dynamics through its role as a tool in the development of contemporary social policy. In addition, the conventional understanding of evaluation as a value-free activity and neutral technocratic tool brings arguments on the practicalities of value-free evaluation in practice, as it operates within social systems that have inherent social meanings and power relations (Taylor & Balloch 2005). In this regard, the practice of development evaluation has implicit issues of power regarding the legitimacy of knowledge, as evaluation has value judgements regarding setting standards around what should be considered as real programme outcomes, what knowledge measures that reality and what values support the evaluation. Chilisa et al. (2016), for example, describe the current trends in development evaluation as ‘the worst instrument of epistemological imperialism’. Evalutaion therefore speaks to power dynamics around who makes decisions and judgements about the realities and relevance of measures to improve interventions.

Currently, African evaluations² are viewed as maginalised, with no power to determine approaches for theorising, designing and ultimately controlling evaluation findings and reports (Uwizeyimana 2020). In this regard, Afrocentric evaluation practice requires shifts in the decision-making environments and building capacity for African evaluators. The principle of ‘powerful for Africans’ in the draft AEPs further elaborates the need for shift in epistemic power dynamics. The principle further outlines other power-related issues to include the need for appropriate, empowering processes, encouraging reciprocity, allowing mutual accountability, enabling learning for useful insights and valuing and strengthening local capacities.

Despite development evaluation being generally viewed as a scientific knowledge generation-based inquiry, it is an inherently political activity (Azzam 2010; Brandon & Fukunaga 2014; Eckhard & Jankauskas 2019). It is therefore critical to understand the roles of various stakeholders in influencing evaluation processes and how such dynamics affect the validity of evaluation results. Additionally, evaluation practice should integrate mechanisms that minimise impacts of stakeholder interests on evaluation design and implementation, as well as the positionality of evaluators to enhance integrity of evaluation results. For example, Morris and Clark (2013:66) point to situations where evaluators are pressurised to misrepresent findings as common occurrences in evaluations. In addition, Azzam (2010:45) outlines situations where stakeholders can dramatically affect how an evaluation is designed and implemented. In line with these power dynamics, Eckhard and Jankauskas (2019) propose incorporation of ‘evaluation stakeholder influence potential’ as a key component in evaluation practice. Evaluation of stakeholder influence is important for the MAE approach for epistemic reasons (the need to ensure appreciation of MAE principles by stakeholders or changes in evaluation culture) and instrumental reasons (to ensure evaluation designs are in line with MAE paradigms). The AEPs call for the evaluation process to include appropriate empowering processes and encourage reciprocity and mutual accountability among various stakeholders involved in the evaluation process.

Evaluation stakeholder influence potential is based on four political resources for influence, which are power for agenda-setting, staff and budgetary resources, access to evaluation results and access to evaluators. In practice, different stakeholders have varying levels of influence for each of these resources; thus, stakeholders have varying capacity to influence evaluation results. According to March and Olsen (1998), an additional source of influence is informal political power that may have influence on evaluation results.

The power dynamics in evaluation invariability touch on issues of positionality. This hinges on African evaluators’ worldviews and the positions they adopt during the evaluation in line with social and political contexts (Foote & Bartell 2011). According to Merriam et al. (2001:411), one’s positionality is understood as ‘where one stands in relation to “the other” in research’. Thus, the positionality of evaluators in the African context will be grounded by their ontological, epistemological beliefs and assumptions about human nature and agency (Grix 2019). This positionality is framed by values and beliefs in line with factors including political allegiance, religious faith, gender, sexuality, historical and geographical location, ethnicity, race, social class and status. A key challenge faced by African evaluators in practising MAE will likely be the need of ‘unlearning’ Western paradigms and having the desire to adapt Western approaches to African contexts. According to McCorkel and Myers (2003:228), a possible strategy to overcome this challenge might be pursuing a strategy that involves ‘a recognition and analysis of how the evaluator’s positionality facilitates specific forms of understanding and impedes others’. This will involve continuous reflections by African evaluators.

Gawlewicz (2014) further proposes the need to incorporate issues of positionality into evaluation practice guidelines. According to D’Silva et al. (2016), possible integration of positionality in enhancing MAE might be achieved through ensuring that evaluation teams are diverse in terms of nationality, gender and other dimensions and that evaluators are conscious of their positions and continuously reflective in practice. This will enhance effective engagement with communities to ensure effective grounding of the MAE within specific contexts. The need for a diverse team is illustrated in the following quotation (D’Silva et al. 2016):

The creation of a knowledge base to support the practice of development evaluation rooted in the MAE paradigm requires a strong indigenous African evaluation capacity. This invariably requires attention towards evaluation capacity development (ECD), that is, unleashing, strengthening, creating, adapting and maintaining evaluation capacities over time (OECD 2006). Effective ECD is therefore a primary source for supplying competent African evaluators for evaluation knowledge generation. According to Tarsilla (2014), ECD in Africa is currently ‘donor-centric’, conducted in an unsystematic manner towards fulfilling the needs of donor-driven projects. This approach to ECD has been identified as being blind in promoting context-relevant learning. Morkel and Mangwiro (2019) further argue that, despite efforts to strengthening ECD in Africa, the generative mechanisms through which ECD strengthens evaluation capacity remain unclear. Another challenge is the lack of a common agreed-upon framework for essential evaluator competencies for ECD in Africa (Morkel & Ramasobama 2017).

In general, most of the practising, prospective and emerging evaluators rely on the content and processes made available at tertiary training institutions. However, there is generally a lack of a harmonised and systematic approach to formal graduate education in evaluation in Africa (Tirivanhu et al. 2020). This scenario has resulted in a situation where evaluation practice is, to a large extent, influenced by social science research approaches. This is mainly because of the research background of most practising evaluators in Africa. The lack of a clear and systematic ECD framework poses challenges to developing the required critical mass of African evaluators to develop the required knowledge repositories for MAE. In addition, deviations are needed from the functional (training to conduct evaluations following the donor-driven evaluation) towards a transformative ECD strategy. The transformative ECD agenda will focus on decolonising and indigenising evaluation practice, coupled with more representation of Africa-based scholars in scholarly knowledge production on MAE. This will address existing epistemic injustices (Chilisa & Malunga 2012).

According to the draft AEPs, evaluation practice should be powerful for Africans, and it should be Afrocentric. These principles invariably call for the practice of an empowerment and emancipatory evaluation agenda that empowers African society through supporting people to improve their lives and enhances epistemic freedom. Empowerment evaluation encompasses the application of ‘evaluation concepts, techniques, and findings to foster improvement and self-determination’ (Fetterman 1994, 2001:3). In this regard, an empowerment agenda for MAE will revolve around strengthening the roles of individuals and communities and ensuring their influence and control over decisions that affect their livelihoods (Coombe 2002). Therefore, an empowerment evaluation process should allow evaluands to oversee their environment (physical, economic, social, cultural and psychological) using available resources (Zimmerman 2000). Empowerment evaluation therefore shifts political and decision-making dynamics through placing decision-making in the hands of community or evaluands. A key principle of empowerment evaluation is the need for promoting collective insights, wisdom and experiences to inform evaluations (Fetterman & Wandersman 2005). In practice, evaluands should be engaged throughout the evaluation phases (conceptual design, data collection, analysis and reporting), and this enhances knowledge utilisation (Fetterman 2001).

The practice of an empowerment approach for MAE should ideally be informed by the 10 principles of empowerment evaluation (Fetterman & Wandersman 2005). These principles are improvement, community ownership, inclusion, democratic participation, social justice, community knowledge, evidence-based strategies, capacity-building, organisational learning and accountability. The principle of improvement should be guided by the desire for African-rooted evaluations to contribute towards social change through a desire to contribute towards positive societal results. The community ownership principle is based on the assertion that evaluands have a right to make decisions about actions that affect their lives. In this regard, evaluators are guided by the fact that programme improvement occurs when the community or evaluands are empowered to exercise decision-making through legitimate authority that drives the evaluation process. The inclusion principle ensures that evaluation encompasses inclusive community ownership with direct participation of key stakeholders in decision-making. The democratic participation principle recognises the capacity of stakeholders for intelligent judgement and action when equipped with appropriate information and conditions, and this enables stakeholder buy-in. According to Fetterman and Wandersman (2005), the practice of democratic participation requires the involvement of stakeholders in decision-making to ensure that evaluation fits the needs and values of evaluands.

The principle of social justice aims at ensuring fair, equitable allocation of resources, opportunities, obligations and bargaining power (Dalton, Elias & Wandersman 2001). Thus, evaluators must take cognisance of social inequities and aim at integrating them into the evaluation design and implementation process to ensure positive social change. The principle of community knowledge ensures incorporation of community-based knowledge, values and wisdom into evaluations. This is very much in line with the thrust of MAE evaluation of promoting epistemic freedom. Empowerment evaluation should embrace local knowledge systems (including tacit knowledge) and appreciate the capacity of evaluands to generate their own solutions. Through the principle of evidence-based strategies, empowerment evaluation should value scientific approaches rooted in African epistemologies as a valid evidence base. This principle is in line with the draft African Evaluation Guidelines’ principle of ‘connected with the world’ (see Table 1). Made in Africa evaluation should appreciate the role of Western knowledge in providing good practices to avoid the risk of totally trying to reinvent the wheel and to build from existing literature or practice – keeping in mind the need for adjusting to African contexts.

The principle of capacity-building is linked with issues of ECD outlined in the preceding section. According to Fetterman and Wandersman (2005), empowerment evaluators should be guided by the belief that allowing stakeholders to learn the basic evaluation steps and skills enhances their capacity to shape and improve their livelihoods. The principle of organisational or community learning must allow evaluators to embrace the importance of continuous learning. African-rooted evaluations should encourage learning and integrate learning in evaluation design and implementation. This may include integrating after learning reflection sessions to bring out learning points from evaluations. Finally, the principle of accountability should provide a basis for evaluation as an innovative accountability mechanism through generation of valid results.

This article explored the practical implications for MAE. The key research question was the following: what are the critical components for MAE development evaluation practice? An exploratory research design was adopted, guided by theoretical constructs from CSH, existing evaluation frameworks, practice guidelines (including the African Evaluation Guidelines and African Evaluation Principles – Standards and Norms), and relevant theoretical and methodological guidelines. Data were collected through secondary literature reviews, as well as expert and experiential knowledge regarding development evaluation practice. The results indicate the critical practice components for MAE to (1) include appreciating sources of motivation as guiding principles of developing modalities for evaluation practice, (2) understanding and integrating sources of power and politics of value judgements in development evaluation practice, (3) developing sources of knowledge and (4) appreciating sources of legitimation (MAE for whom and implications for practice). Methodologically, the research recommends adoption of African-rooted paradigms, including relational approaches and tools grounded in African worldviews, social systems and values. Made in Africa evaluation should mainstream an empowerment evaluation approach that aims at contributing towards positive social change and promoting epistemic freedom of African evaluators.