Invited Paper

Review of Goldman’s book through the lens of the work of Sulley Gariba

Grace K. Igweta, Nikki Zimmerman

Received: 29 Jan. 2024; Accepted: 13 Mar. 2024; Published: 20 June 2024

Copyright: © 2024. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Introduction

One way to honour the memory of Sulley Gariba is to reflect on contemporary discourse on issues that were central to his work and contributions. One of these issues is evaluation and its contribution to development outcomes (see Anyidoho & Gariba 2015; Gariba 1998; Gariba & George Keith 2005; Jackson & Gariba 2002).

In this article we use two of Sulley Gariba’s works to lay the foundation for the review of the book edited by Ian Goldman and Mine Pabari on using evidence in policy and practice and lessons from Africa.

The first is the case study of the Northern Region Rural Integrated Programme in Ghana in the 1980s in which Gariba (1998) illustrates use of participatory impact evaluation. He describes purposes of the evaluation and examines the extent to which using participatory evaluation method influenced the programme. It can be argued that what Gariba (1998) demonstrated in this work is what has come to be known as developmental evaluation (Patton 2010). This was a kind of evaluation that has the potential to influence implementation of programmes without waiting several years to evaluate only to find that wrong interventions were implemented or right interventions were implemented the wrong way.

The second is three cases of social movements in Ghana, Egypt, and Nigeria to investigate learning within collective action movements (Anyidoho & Gariba 2015). They present three typologies of learning – learning in struggle, learning through struggle, and learning to struggle. These are ways through which those involved in interventions and the wider citizenry learn about contexts, possibilities, and strategies of collective action (Anyidoho & Gariba 2015:4). The study showed the importance of learning both within and from interventions and evaluation’s role in promoting learning. Evaluation processes, when participatory, allow space for those involved in implementation of the subjects of evaluation to take account of what has been learnt while also providing those outside the implementation spaces the opportunity for learning (Anyidoho & Gariba 2015:24).

Aim of the book and the context

The global discourse on evidence-based policy (EBP) is vibrant. Some argue that despite critical literature emerging, it has become a movement in itself (Cairney 2016; Parkhurst 2017; Simons & Schneidermann 2021). As Paul Cairney observes in his foreword to the edited volume, the EBP discourse is largely by and from a small number of researchers in the countries of the global north. He points out that the book seeks to correct this by giving voice to African experiences and sharing lessons from those who demand and use evidence from across the continent. The editors state that the book was written to ‘improve understanding of how best to facilitate use of evidence to improve policy and practice and facilitate social outcomes in an African context’ (Goldman & Pabari, 2021a:238).

Unlike the African context within which Gariba (1998) was writing – that of undifferentiated gloom and doom narrative about Africa that he considered unjustified – Goldman and Pabari (2020) write in more optimistic times, which they refer to as ‘a period of self-discovery’. This is a period of economic growth on the one hand and the struggles of translating that growth into opportunities for all citizens to prosper, on the other.

Authors’ approach to the topic

What readers get in this book is a distillation of lessons from individual and organisational experiences in using evidence in a variety of contexts. The book uses a case study approach, with eight cases from six countries and the Economic Commission of Western Africa States (ECOWAS). The case studies are written as ‘evidence use stories’ by researchers and policymakers working together to explore their evidence journeys and identify factors that enabled or hindered use of evidence in their context.

The book has 13 chapters arranged in three parts: four introductory chapters, eight case study chapters, and a concluding chapter. After Goldman and Pabari (2021a) introduce the contexts and the cases in Chapter 1, the editors discuss theories and concepts of evidence-informed policy and practice in Chapter 2 (Goldman & Pabari 2021b). Having acknowledged the debates on whether it is EBP, Evidence-Informed Policy Making (EIPM), Evidence-Informed Decision Making (EIDM), or Evidence Informed Policy and Practice (EIPP), the authors settle on EIPP (Goldman & Pabari 2021b:14). This choice is not trivial given the literature on politics of evidence and policy (Cairney 2016; Parkhurst 2017; Simons & Schneidermann 2021). Unlike some advocates of EBP who consider the definition of EBP to include the research method through which evidence is generated (see Baron 2018 who equates EBP to generating evidence through randomised control trials), Goldman and Pabari (2021a:2) consider evidence that comes from a variety of sources including scientific research, evaluations, indigenous knowledge, administration data, surveys of public opinion, and so on. Readers will appreciate the authors’ healthy realist point of view that policymaking is not wholly rational and does not take place in a vacuum. In the complex social world, policymaking is an ongoing process, influenced and informed by several factors, with evidence, no matter the methods of its generation being only one factor. It is refreshing to see a book that acknowledges this fact, and still does a good job of showing how various types of evidence can be used to inform decisions.

In Chapter 3, Langer and Weyrauch (2021) present the analytical framework (shown in Figure 1), which is comprehensive yet easily understandable. It provides a lens for categorising, analysing, and synthesising lessons from activities that support use of evidence (Langer & Weyrauch 2021:39).

FIGURE 1: Analytical framework.

In developing the framework, the authors build on existing research and frameworks quite well, specifically integrating the Science of Using Science (Langer, Tripney & Gough 2016) and Context Matters (Weyrauch, Echt & Suliman 2016) frameworks. The result is a framework that pays attention to context and simplifies the process of use (from demand, supply, to use) from a behavioural change lens. Importantly, the authors advocate for intended users (researchers, policymakers, EIDM practitioners) to use the framework as it befits their context, adapting it as they see fit. The authors do just this in Chapter 13 based on learnings from the case studies, with some important adaptations. However, while promoting the applicability of the framework, the authors heavily caveat its use – by belabouring the point on what it is not. This leaves the reader wondering whether they have doubts as to the versatility of the framework. Nevertheless, the framework holds much value, with further adaptations and complementary tools to help translate learnings generated by the framework into practice.

The case studies

The content chapters generally follow the same approach. They lay out the context, explain the nature of the evidence, and trace the ‘evidence journey’ of how the evidence was generated and used. Each chapter discusses factors that enabled and hindered use of evidence and concludes with reflections and lessons. In taking this approach, the authors make it an easy read. However, the depth of analysis and what aspects are emphasised differ from case to case.

In Chapter 4, Goldman et al. (2021) present findings from research on performance monitoring and evaluation culture in three countries – Benin, Uganda, and South Africa. The authors conclude that the three countries have elaborate planning and monitoring systems and established National Evaluation Systems (NES). One of the key enablers for use of evidence was political will and evidence demand because of the location of the M&E function within the government structures, which gave it authority. Another enabler was donors’ reinforcement of the NES instead of operating parallel systems. Evaluation evidence was found to be used more ex-post rather than during the life of interventions. Some key hindrances to evidence use were staff turnover, especially at the leadership level, limited ownership, silos, and weak involvement and linkage with civil society.

In Chapter 5, Pophiwa et al. (2021) share a case study of evidence use in the department of basic education in South Africa observing that the department has been a pioneer in using evaluation, research, and data. The case uses two evaluations undertaken by the department to demonstrate the importance of an institutionalised national system and evidence champion or broker in encouraging evidence use within departments.

Chapter 6 is about use of evidence from a diagnostic review of the state’s response to violence against women and children in South Africa (Amisi, Buthelezi & Magangoe 2021). This case study illuminates on what enables and hinders evidence use in addressing complex multisectoral issues. As with the education case study in chapter 5, the existence of a functioning NES was an enabler in addition to strong knowledge brokering by evaluation units within relevant departments and creation of spaces for inter-sectoral dialogues.

In Chapter 7, Kawooya et al. (2021) discuss lessons from use of evidence from an evaluation of a public procurement system in Uganda by the Prime Minister’s office and the public procurement development authority with support from the World Bank. The evidence was used to inform revision of procurement thresholds and flexibility for sectors that need specialised procurement. The evaluation demonstrated the importance of regularly reviewing and updating regulations, standards, and guidelines.

Chapter 8 is a case of knowledge brokers in Uganda rapidly responding to policy evidence needs through a Rapid Response Service at Makerere University (Kawooya et al., 2021a). Using three cases where evidence was provided through this service, the authors show three uses of evidence in practice: Conceptual use where evidence was used by decision makers to stimulate debate and clarify issues and concerns regarding food fortification; Symbolic use where decision makers used evidence to clarify some concerns that had already been raised about the use of certain medication; and Instrumental use where evidence was directly used to implement actions that improved efficiency of a health service.

In Chapter 9, Kouakanou et al. (2021) show the potential and challenges of evaluations to positively influence reforms in the agriculture sector in Benin. Like in the South African cases, this is set in a context with a functioning NES, which anchored the evaluation of the agricultural sector development policy and was one of the enabling factors. While the authors acknowledge that the evaluation was not used instrumentally, they show that it was conceptually used to clarify and understand the needs of the sector. Apart from the NES, other key enabling factors were the opening of policy space to non-state actors and donor conditionalities.

Chapter 10 explores the role of the parliament in policymaking through citizen engagement and public participation in Kenya using the case of World Life Conservation and Management Act 2013 (Pabari et al. 2021). Tracing a 16-year journey that brought the act into being, the case study shows how evidence collected from the public and other stakeholders was used to inform changes to the contents of the act (instrumental use) and how experiences from the process were used to strengthen tools and approaches to public participation as well as in rebuilding trust and relationships between government and civil society (process use). This case study is a lesson in complexity of policymaking processes in Africa. It brings to focus one of the core differences between policymaking in the countries of the global north and those of the global south – the presence and critical role of external non-state actors including donors and non-governmental organisations. The enabling and hindering factors illuminate the importance of this factor, including risks of non-state actors with resources swaying the discourse based on their interests and ideological positions.

In Chapter 11, Smith et al. (2021) explore the role of civil society in generating evidence to inform improvements in sanitation service delivery in Ghana. They examine how different stakeholders have used evidence from the District League Table (DLT) – a tool developed by a non-government organisation that provides assessment of social development and ranks all districts in Ghana. The DLT is intended as evidence for enhancing performance in the sanitation sector. What this case shows is the power of citizen-led evidence processes that have dual purpose – putting pressure on government actors to improve service delivery while encouraging and inspiring the governed (citizens) and the governing (public actors) to work together for the betterment of society.

In Chapter 12, Mane et al., 2021 share experiences of generation and use of evidence through action research on application of the World Health Organization (WHO) Framework Convention on Tobacco Control (FCTC) within the ECOWAS. In each country, tax rates were modelled and discussed with stakeholders through workshops and evidence was documented in country profiles, regional synthesis, and a position paper. A new directive on taxation within ECOWAS was adopted based on the position article while Senegal used the country profile to develop a new law on tobacco control. Besides these instrumental uses, the process contributed to increased interest in further research on this topic (process use), better understanding of the issue (conceptual use) and helped to counteract tobacco lobbying (positive symbolic use).

The editors bring everything together in Chapter 13, guided by the framework (Goldman & Pabari 2021c). They reflect on which mechanisms were at play in enabling evidence use, the types of uses, and what lessons emerged. Acknowledging the complexity of weaving evidence into policymaking processes and practices, the authors notice the importance of quality facilitation and knowledge brokering that spans the supply and demand sides of the evidence equation. The use of the evidence journey metaphor is informative in this regard as the facilitation starts long before the evidence is to be generated and used. The authors note three core sources of demand for evidence – national evaluation systems, donors and or development partners, and civil society or citizens. What enables or hinders use of evidence differs based on the sources of demand and the context within which the evidence is demanded and generated. The authors conclude that the key to facilitating successful use of evidence to improve policy includes understanding of context, involving stakeholders continuously, ensuring demand for evidence and appropriate supply, using change mechanisms, building capabilities and motivation, establishing buy-in at high levels, and exploiting opportunities within the policymaking process.

Significance and contribution

This book is a critical part of Africa’s contribution to the burgeoning literature on EBP. Its strength is in the richness and diversity of the case studies and sources of evidence, deliberate conversation between researchers and policymakers in telling the evidence stories and doing so through an explicit and easy to understand analytical framework. Readers will, we hope, appreciate the authors for steering clear of the debates about evidence hierarchies and for being pragmatic in showing what can be achieved if policymakers, researchers and practitioners work together to use evidence in broad sense in making timely decisions.

Overall, this book provides a good reading for people who wish to understand the potential role of evidence in improving decision-making, service delivery and development outcomes and contribute to realisation of that potential through generation, dissemination, and use of evidence from different sources and perspectives. Each case study serves as an exemplar from which policymakers, practitioners, and researchers working in that sector or context can learn from and build on. Students of public policy and evaluation will find themes and sub-themes in this book worth exploring.

The book makes an important contribution in managing expectations on the role of evidence, especially in the short-term. While practitioners (and the framework used in this book) assume evidence use will contribute to better development outcomes, instances of evidence use were predominantly process or procedural, with rare instances of changes in budget allocations, for example. The use of the ‘evidence journey’ metaphor and the case studies manage these expectations.

One improvement the editors should consider is to give more thought and space to evidence generation and the interplay between supply and demand in the conceptual framework. The authors start from the user pull rather than evidence generators push, arguing that policymakers and practitioners are protagonists seeking evidence to inform their practice rather than passive recipients of research (Langer & Weyrauch 2021). While we agree with this argument, we think that the analytical framework can be enhanced by being explicit about the interplay between supply and demand and acknowledging that supply and demand are not only part of organisational internal context as they can originate externally. This is important for two reasons: to avoid the risk of assuming only on-demand evidence is used and to acknowledge that in some cases evidence availability precedes demand. If policymakers only have access to and use evidence that they demand, we may miss opportunities to learn from evidence generated because of other triggers including academic research, media, or evaluation of citizen’s collective action as shown by Anyidoho and Gariba (2015).

We hope there will be future editions of this book to share more evidence use journeys from Africa to continue contributing to sharing lessons on EIPP across contexts.

Acknowledgements

The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.

G.K.I. prepared the first draft and N.Z. reviewed and edited it. There were discussions between the authors on the overall tone of the article before G.K.I. finalised and prepared it for submission.

This article followed all ethical standards for research without direct contact with human or animal subjects.

Preparation of this review article received no funding from any public, commercial, or not-for-profit sectors.

Data sharing is not applicable to this book review article.

The views and opinions expressed in this article are those of the authors and are the product of professional research. It does not necessarily reflect the official policy or position of any affiliated institution, funder, agency, or that of the publisher. The authors are responsible for this article’s results, findings, and content.

References