Dr Sulley Gariba was a pioneer for global and African evaluation within the EvalPartners family. He was a strong advocate for Making Evaluation Our Own and Made in Africa Evaluation. This article pays tribute to Dr Gariba by examining the growing movement to improve the cultural relevance of evaluation and in his words, make evaluation our own in the Asia Pacific by raising awareness and increasing the application of culturally responsive evaluation (CRE) and culturally responsive Indigenous evaluation (CRIE).

In this article, culture is broadly defined as ‘the shared living experiences of people’, including groupings based of ethnicity, religion, class, age, gender, sexual orientation, disability, organisations and institutions (Aotearoa New Zealand Evaluation Association [ANZEA] 2011). The intersectionality between culture, evaluation and values has been described this way:

This article describes examples across the region where localised values inform the design and conduct of evaluation. There are many terms used to describe this type of process including CRE, CRIE, localising evaluation, evaluating using local world views or local wisdom, culturally sensitive evaluation, made in Asia evaluation or made in the Pacific evaluation. In this article, the terms CRE and CRIE are primarily used, but there are also references to the other terms listed here.

Culturally responsive evaluation and CRIE both contest the assumption that Western-derived evaluation methods can be universally applied and argue that evaluations need to ensure the cultural relevance of the methods being used (Chouinard & Cram 2020; Howe 1994). CRIE, which is described by Waapalaneexkweew (Bowman, Mohican/Lunaape) and Dodge-Francis (2018) as a ‘partner to CRE’:

This article describes examples of CRE and CRIE from across the sub-regions of South Asia, Southeast and East Asia and the Pacific in order to demonstrate the diversity of forms that CRE and CRIE can take. Some of the examples recognise pluriversalism or the interconnectedness between Western and/or colonial and community and/or Indigenous cultures and use hybrid approaches that respect and intermingle Western and localised evaluation approaches and methods (Jordan & Hall 2023). In this article, the analysis primarily draws upon the use of CRE in international development practice in the region.

A movement to strengthen and localise evaluation practice by increasing the application of CRE and CRIE in the Asia Pacific is underway. Such a movement strengthens evaluation practice by ensuring that communities involved in an evaluation are empowered to lead and/or authentically participate in the process and that the evaluation is driven by the values and priorities of that community. In doing so, the literature points out that such a movement contributes to decolonisation and self-determination through, ‘valuing, reclaiming, and foregrounding Indigenous voices and epistemologies’ (Darder et al. 2014). In the literature on CRE and CRIE, decolonisation refers to a reversal of:

In the international development sector, adoption of CRE and CRIE also redresses postcolonial and other power disparities that have resulted from decades of practice. The literature describes how much of this practice has reinforced Western countries’ positions of power, whereas CRE and CRIE enable significant agency for the communities involved to seek out development benefits that are in their own interests (McGregor 2009; Wilson 2019).

In this section, good practice examples of research and implementation of CRE and CRIE in the Pacific, Southeast Asia and South Asia are described. These are neither intended to paint a comprehensive picture nor to be representative of the region as a whole.

There are 15 independent nations in addition to tens of thousands of islands, islets and atolls in the Pacific region. There is strong practice of identifying and applying research and evaluation that embody the culture, history, traditions and relationships of diverse communities and countries in the region. There is also significant literature on the processes of conceptualising and applying such practice.

One localised approach to research and evaluation that is well known in the Pacific is talanoa. Rooted in a long oratory tradition, talanoa has emerged as approaches to research in countries in the Pacific that involve ‘engaging in dialogue with, or telling stories to each other absent [of] concealment of the inner feelings and experiences that resonate in our hearts and minds’ (Halapua 2008). Talanoa is a concept recognised in many Pacific countries, including Fiji, Samoa, Solomon Islands, Niue, Hawai’i, the Cook Islands and Tonga but in each context within and between these countries it is conceived and applied differently. To an observer unfamiliar with talanoa, it could seem like it is merely a Pacific form of ‘informal open-ended interviews’, but talanoa is actually a multidimensional process involving empathtic understanding and exchange between the participants and the researcher. Vaioleti describes talanoa research as, ‘holistically intermingl[ing the] researchers’ and participants’ emotions, knowledge, experiences, and spirits’ (2006).

Conducting talanoa research and employing it during evaluations involves a deep understanding and practice of empathy between the researcher and the participants (intersubjective empathy), which is conceived as attachment to living kin, deceased ancestors and land, and how a life is lived, which in turn can influence a person’s health or ecological impacts such as the success of a crop. This requires moving away from a Western-derived conceptualisation of empathy, which is subjective and could be viewed as impersonal and reductionist. Subjective empathy is one-directional with the researcher attempting to ‘walk in the shoes’ of the participant, albeit through a passive ‘subject-to-object’ relationship. In literature about talanoa, it is argued that if research methodologies enable intersubjective empathy, then they ‘bring us closer to understanding others’ emotions, feelings and attitudes [and] provide us with a more complex and intimate understanding of cultural and social forms’ (Farrelly & Nabobo-Baba 2014).

‘Our research methodologies must reflect the knowledge making and knowledge sharing of our participants, not the other way around’ (Farrelly & Nabobo-Baba 2014).

The literature talks about how Talanoa research methods are decolonising research and evaluation by reflecting local world views and values throughout the process. Such methods are also subverting the power imbalance between researcher or evaluator and participants that often results using Western research methods by building a shared process involving knowledge, sensation and emotion. When research findings truly reflect the lived experience, values and aspirations of participants, then by extension, policy and practice informed by this research can also be decolonised.

The Asia Pacific region is home to 99% of the world’s Buddhists, who mainly live in Southeast and East Asian countries, such as Cambodia, Thailand, Myanmar, China, Vietnam, South Korea, Japan, Sri Lanka, Singapore and Taiwan. Buddhism is based upon the teachings of Siddhartha Gautama, or Buddha, in India 2500 years ago and remains a significant global influence today (Berzin 1996). Research in the region by Dinh, Worth and Haire (2019) sought to identify how a commonly used Western-derived evaluation approach – the most significant change (MSC) technique could be adapted to take into account a Buddhist world view.

Most significant change is a form of participatory monitoring and evaluation that involves the collection of significant change stories emanating from a grassroots level and the systematic selection of the most significant of these stories by panels of designated stakeholders or staff (Davies & Dart 2005).

In one example from this research, in Buddhist teaching, there is the principle of impermanence. This principle describes the world as in a constant state of change in which everything arises, stays in a state of flux and then passes away. Most significant change is a dynamic and adaptive technique, which, in this instance, is quite compatible with a Buddhist world view as it recognises that change is continuous and captures stories of change as points in time.

However, Buddhism incorporates concepts of causality or ‘how things happen, how change occurs, how events relate’ that are quite distinct from those used in most Western-derived evaluation methods (Macy 1978). The Buddhist premise of causation is that cause and effect are interdependent in the past, present and future (Macy 1991). In Buddhist teachings, this means that cause and effect co-arise together, and they are recursive, not unidirectional (Bukkyō Dendō Kyōkai 1966). This means that what is originally considered to be a cause could come to be considered as an effect, and what is originally considered to be an effect could come to be considered as a cause (Russon 2014). The research suggested a modified approach to MSC, involving the use of a story collection template that took into account these Buddhist teachings and could encourage participants to identify multiple causes to an effect and how the effect has simultaneously influenced the cause.

This research is part of a wider body of work on culturally responsive evaluation, which looked at other Southeast Asian world views and how they could be applied to evaluation. In some cases, these world views were largely compatible with Western-derived evaluation approaches and methods and in other cases there were stark contrasts. The research argued that in ensuring an evaluation method or approach is appropriate for a context, firstly the ontologies and epistemologies of the world view(s) of the community participating in the evaluation as well as those of an evaluation method or approach being considered need to be thoroughly understood. Any incongruities between these world views can then be reduced or eliminated by modifying the method or approach. The research suggested that a similar hybrid approach, which recognises pluriversalism, or the interconnectedness between Western/colonial and community/Indigenous cultures (Jordan & Hall 2023) could be used to reflect multiple world views in an evaluation method. It argued that in the complex, globalised and dynamic societies in which evaluators now work, such hybrid methods are very much needed (Dinh, Worth & Haire 2021).

A study of the Bengali practice of adda in India, Bangladesh and the Bengali diaspora explored the utility of this practice as a localised approach of participatory evaluation. Adda is defined as ‘a semi-structured and informal congregation of folk who by and large belong to similar world view’ (Pathak 2022). There were three central characteristics of an adda that were described. Firstly, that despite the fact that those who congregate hold by and large the same world view, the adda is paradoxically a space in which views are contested. This contestation occurs through assessment of the actions and behaviours of each other, the efficacy of institutions and interpersonal actions. Secondly, assessment is conveyed through particular forms of expression including, ‘ridiculing, provoking, gossiping, the hilarity of expressions, and assessing with judgmental tenor with an iota of shared empathy’ (Pathak 2022). Thirdly, discussion centres on personal-material benefit rather than ideals or the interrelationship of spiritual or physical dimensions.

The practice of adda is rooted in the ancient Indian philosophy of Lokayata. This philosophy promotes a rationalist, empiricist, material means of verification (or evaluation) based upon what is seen and shown, rather than any spiritual or metaphysical dimensions. What is seen and shown includes both cognition of intellect and reason as well as emotion and sentiment.

In considering adda as a particular world view of participatory assessment, Pathak observes that in conceptions of self, Lokayata merges intellect with emotions and intuition with empirical observation. It also embodies the capacity of the self in making choices and decisions. Thus, the evaluator (self) involved in an assessment of others would be ready to ‘converse and quarrel at once’ as well as to self-evaluate (Pathak 2022).

The study observes that other philosophies in South Asia give rise to similar forms of assessment. Thus, Pathak finds that adda not only provides potential for a participatory evaluation approach that is rooted in Bengali world views and philosophies but also that similar evaluation approaches could be derived from other philosophies in South Asia.

These examples demonstrate that there is good practice and a growing body of literature on CRE and CRIE in the Asia Pacific. There is also a need to keep building this body of literature in order to share experiences in applying CRE and CIRE. This building of an evidence base and sharing of experience is important to strengthen the understanding and capacity of evaluators in the region in CRE and CRIE and also to demonstrate the importance and value of such approaches to commissioners of evaluation.