Book Review

Evaluation in and for democracy: Perspectives from South Africa

Book title: Democratic evaluation and democracy: Exploring the reality

Author: Donna Podems

ISBN: 978-1681237886

Publisher: Age Publishing, 2017, R463.60*
*Book price at time of review

Review title:
Evaluation in and for democracy: Perspectives from South Africa

Reviewer:
Zenda Ofir¹

Affiliation:
¹School of Public Leadership, Stellenbosch University, South Africa

Corresponding author: Marius Nel, zenda@zendaofir.com

How to cite this article: Ofir, Z., 2018, ‘Evaluation in and for democracy: Perspectives from South Africa’, African Evaluation Journal 6(1), a305. https://doi.org/10.4102/aej.v6i1.305

Copyright Notice: © 2018. The Authors. Licensee: AOSIS. This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Introduction

This timely, relevant and useful book should be on the bookshelf of evaluators and students of evaluation who are serious about evaluation, who appreciate perspectives beyond the narrow and technical, and who are interested in the role of evaluation in cultivating democracy.

The title draws one in: in our profession we need in-depth and nuanced explorations of challenging realities, and even more so of the realities in the Global South. We are in an era where democratic models of government display remarkable weaknesses, and alternative models display unanticipated strengths. At the same time the evaluation profession around the world is dynamic and increasingly in demand, yet facing extraordinary challenges. At no other time would it have been more appropriate to engage with this complex topic in practice through a case study of one of the most intriguing evolving democracies in the world.

Exactly at the right moment, Donna Podems brings us a collection of 14 thought-provoking chapters that dissect from different perspectives the interplay between democracy and evaluation in the real world of South Africa’s young democracy. The eclectic group of 19 co-authors, nearly all from the South African evaluation community, presents a set of engaging arguments and insights in response to the question that directed the structure and content of the book: ‘What role, if any, does evaluation play in strengthening those countries that are still committed to, but struggling with, democracy?’ To echo how the first chapter positions the book, the following question fits: are democratic evaluation and a new democracy ‘acquaintances, adversaries or allies?’

Structure and content

The book is structured as a case study that is intended to inform evaluation teaching and practice. This practical approach is reflected in the flow and presentation of the chapters, and in the free-flowing reasoning in each case without any boundaries imposed by a particular framework or analytical approach.

Positioning the case

The first three chapters give a broad, international overview of key issues, providing a lens through which the rest of the book can be viewed: definitions and brief explanations of important concepts (Podems, Chapter 1); a discussion of the role of power in governance and its implications, based on three political configurations applied in five African countries (Porter, Chapter 2); and one of the only chapters focusing on non-government-driven experiences, a very practical examination of the imperatives and challenges in applying democratic evaluation on the ground in international development contexts (Levine, Chapter 3).

One could argue that the key concepts and lines of argument in these chapters should have been used in the rest of the book to help frame, ground and synthesise important insights. Some might consider the absence of such a framework as a drawback – and in some ways it is. But instead of telling the reader what to think, different concepts and perspectives are provided to inspire own interpretations as the reader moves through the book. This makes for a refreshing approach in tune with the purpose of the book.

Focusing on South Africa

From Chapter 4 onwards, the focus narrows to South Africa. In Chapter 4, Abrahams provides a primarily descriptive, useful narrative of the evolution of the South African Monitoring and Evaluation (M&E) Association and the issues it has faced in trying to maintain a delicate balance between engaging with the South African government as the key driver of evaluation in the country and maintaining the necessary independence from it. Although not specifically discussed, issues around power highlighted in Chapter 2 are particularly relevant here. Chapter 4 also touches on the very important issue of which capacities and systems are in place – or should be in place – to prevent capture of evaluation by the state while avoiding what could be destructive animosity between the state and the rest of the evaluation community. This important issue deserves more scrutiny, evidence and analysis; it is of great importance to all Voluntary Organisations for Professional Evaluation (VOPEs) around the world and demands highly responsible and ethical action as well as appropriate capacities in both the government and the relevant VOPE.

It is useful to consider Chapter 4 in conjunction with Chapter 11 (Beney). The latter provides a highly relevant and empathetic discussion of the importance of agency in the institutionalisation of evaluation in a country, in particular among parliamentarians, the ‘institutions supporting democracy’ and civil society. Beney acknowledges that lack of information and evidence limits his discussion of these important issues at this point, but for me this chapter illuminates one of the main overall takeaways from the book – the difficulty of ‘massifying’ evaluative practice in support of democracy, and in particular in institutions that matter.

The context for evaluation from a national perspective

Chapters 5 (Cloete) and 6 (Goldman) provide important conceptual as well as contextual information that is imperative for locating the discussions in the chapters that follow. Both of these authors are experienced and highlight vital aspects of the evolution of government and evaluation in South Africa and, based on these, provide thoughtful insights from different perspectives on the interplay between democracy, government and evaluation. Their candid analyses are appreciated; they highlight not only significant progress and advantages but also important shortcomings and challenges in the current approaches to evaluation by government.

Main takeaways from both chapters are at the same time encouraging and discouraging: good progress through commitment and expertise, but major constraints inherent in the nature of the systems and society within which government M&E systems have to be embedded. Can some of these ever be overcome to meet the theoretical ideals of democratic evaluation in service of democracy?

Government-driven evaluation in practice

The four chapters that follow (Amisi and Vawda, Chapter 7; Williams and Scherman, Chapter 8; Jakoet and Wyatt, Chapter 9; and Troskie, Kelly and Mandondo, Chapter 10) are from a practical on-the-ground perspective the most valuable. They discuss experiences and lessons in doing evaluations commissioned by national and provincial departments in different sectors around South Africa. The occasional overlap in topics covered does not detract from the content: each chapter reflects from different vantage points on the nature of democracy in South Africa, the particular context for the evaluations, and successes and challenges. Similar to Chapters 5 and 6, they provide thoughtful and candid narratives that reinforce perceptions of less than satisfactory to good progress amidst formidable challenges. The description of the Western Cape Department of Agriculture’s efforts to institutionalise and embed evaluation in its culture (Chapter 10) is particularly encouraging for the future; it is at this level that evaluation has its greatest potential to contribute if service delivery can be improved through evaluative thinking and practice.

Some of the evaluations strived to be of the democratic kind; others did not or could not attempt this design. An in-depth comparative analysis and synthesis of the approaches, experiences and context-sensitive influencing factors and the implications for evaluation systems and for democratic (or non-democratic) evaluation in South Africa would have been intriguing; later chapters only touch on the rich pickings that can be gained from such reflections. Again, given the teaching and learning orientation of the book, it is left to the readers to reflect on the content and draw conclusions from the rich descriptions in each chapter.

Synthesis

Chapters 11–13 are the most interesting from a national and international learning perspective, as they provide overall syntheses with rich insights for the question that shaped the book. Chapter 11 (Beney) makes a case for the need for multiple centres of evaluation production and use in the executive, in democratic institutions and in civil society, yet reminds us that evaluation as ‘technical practice’ can in the end only contribute to democracy in a partial and contingent manner.

Chapter 12 by Fraser and Rogers is a valuable contribution and addresses many concerns that arose as I read the preceding chapters. This important chapter is rich because the authors provide a considered critique of the current national evaluation system and the way in which evaluations are commissioned and conducted. They rightly emphasise that it is necessary to go beyond the production of evaluation findings and managerial adoption of recommendations, and that inclusion, dialogue and deliberation have to date been underplayed in the commissioning and conduct of evaluations. They also provide more than any of the other authors a critical analysis of the South African democratic context today and highlight pitfalls and the need for nuanced approaches with specific reference to democratic deliberative evaluation in support of democracy under these conditions.

In Chapter 13, Benjamin has been entrusted with the difficult task of weaving together key arguments across all chapters. I would have liked a final synthesis more robustly embedded in and considerate of the South African context and hence more nuanced against context – especially as the authors of each chapter defined and discussed democracy in South Africa and the role of (democratic) evaluation from different vantage points. Benjamin’s synthesis is insightful, although a succinct summary of all the key conclusions would have made it more useful. In the end, it is still not quite clear whether democratic evaluation and the new democracy in South Africa are ‘acquaintances, adversaries or allies’. The chapter ends with a set of questions for future reflection that can help deepen the understanding based on what the editor calls modestly ‘only the beginning’ of filling an important gap in our understanding.

Utility of the book

In the final chapter (Chapter 14), Podems highlights the utility of the book for the engaged evaluator and for teachers and students of evaluation by providing a set of perceptive case study questions aimed at enticing the reader to connect deeply with the content. They provide an opportunity for the reader to systematically extract key lines of argument and to consider the implications for (democratic) evaluation of the different interpretations of context, concepts and experiences. This is a valuable contribution that expands the relevance and utility of the book significantly.

Some reflections

Firstly, the discourse throughout all the chapters indicates that the development of an effective national system should take place in stages, given how difficult it is to build an ideal system with the required qualities. This is especially important for a country at South Africa’s stage of development, and with its particular challenges as a highly unequal and violent society with a relatively poor level of education. Much of the value of the book lies in its illumination of the frequent divide between evaluation theory as ideally applied and the realities of practice – as well as the need for the challenge of ‘massification’ of evaluation thinking and practices if evaluation has to fulfil a truly developmental role in a country like South Africa.

Secondly, the book highlights the complexity of evaluation in societies that have not had the benefit of many decades or centuries of prosperity with strong and mature institutions, diverse capacities and highly educated societies, and where governments have the challenge of ensuring that many moving parts lead to a positive national development trajectory. These are massive obstacles in the effective functioning of an emerging M&E system and make the analysis of the South African experience even more pertinent and useful, in particular for countries in the Global South.

Thirdly, the rich content in each chapter provides opportunities for reflection by the reader, including about important issues that have not been covered in any of the chapters. Examples are, among others, the following: (1) the risk that democratic evaluation can cast a veneer of credibility over the evaluation and its findings while purposefully misrepresenting or selectively using evidence, thus using the values underlying democracy to undermine it; (2) the immediate risk that poorly designed or conducted evaluations resulting from inadequate capacities can at best prevent useful findings and at worst wreak havoc in fragile societies; (3) lack of focus on actual or potentially negative consequences of democratic evaluation; and (4) the extent to which evaluation as currently conceived and done in South Africa can actually assist in helping to resolve some of the most intractable development problems in a violent and unequal society.

Fourthly, ‘democracy’ means different things to different people in different contexts. It is commendable that the first chapter considers whether democracy is indeed a universal value, but the discussion needs to be taken further. It is necessary to account for nuanced differences in interpretation and application based on different societal philosophies and values around the world, including in systems that appear to be undemocratic. The chapters are imbued with values and perspectives derived from Western notions of democracy that have also been taken on by African intellectuals steeped in Western frameworks. This means that there has been no in-depth interrogation of how the particular nature and values of a society could – and indeed do – direct democracy as well as evaluation approaches such as democratic evaluation.

Fifthly, difficult choices have to be made when there is conflict between a government’s long-term strategic vision with hard trade-offs and the short-term, almost inevitably self-centred interests of a community that might be subject to sacrifices inherent in such trade-offs. This is particularly pertinent in countries coming from a low base of development across many sectors. The limitations of democratic evaluation under such circumstances and its contrast with other forms of evaluation could have been more deeply interrogated in certain chapters.

Conclusion

We should celebrate the rich content in this book that highlights the value of deep engagement with evaluation in the Global South from the perspective of both evaluation theory and practice. It will stimulate further intensive and rigorous examination of the contributions and value of different types of evaluation for countries at different and challenging stages of development. I trust that the book will become required reading for all those interested in how evaluation is unfolding in democratic contexts and in countries where evaluation is still in the process of being institutionalised. By engaging forcefully with this important yet complex topic, Podems and her co-authors have made an invaluable contribution to the profession.