Commentary on Country-Led Monitoring and Evaluations Systems, Better Evidence, Better Policy, Better Development Results
Book Title: Country-Led Monitoring and Evaluations Systems, Better Evidence, Better Policy, Better Development Results
Authors: Marie-Helene Adrien;
Petteri Baer;
Michael Bamberger;
Oslvaldo Feinstein;
Enrico Giovannini;
Denis Jobin;
Megan Grace Kennedy;
Oumoul Khayri Ba Tall;
Jody Zall Kuzek;
Hans Lundgren;
Keith Mackay;
Finbar O’Brien;
Kris Oswalt;
Robert Picciotto;
Nicolas Charles Pron;
Jean Serge Quesnel;
Ray Rist, Jim Rugh;
George Sakvarlidze;
Marco Segone;
Daniel Vadnais
Publisher: UNICEF
Open access, link to book: “http://www.unicef.org/ceecis/resources_10597.html” www.unicef.org/ceecis/resources_10597.html
Reviewer: Inoussa Kabore1
Affiliation: 1UNICEF Regional Office for West and Central Africa, Senegal
Postal address: P.O. Box 29720, Dakar-Yoff, Senegal
How to cite this book review: Kabore, I., 2013, ‘Commentary on Country-Led Monitoring and Evaluations Systems, Better Evidence, Better Policy,
Better Development Results’, African Evaluation Journal 1(1), Art. #46, 3 pages.
http://dx.doi.org/10.4102/aej.v1i1.46
Copyright Notice: © 2013. The Authors. Licensee: AOSIS OpenJournals. This is an Open Access article distributed under the terms of the
Creative Commons Attribution License, which permits unrestricted use, distribution,
and reproduction in any medium, provided the original work is properly cited.
Background
Before starting my review, I thought it would be important to first present a quick synopsis of the book, to help readers who may have not had the opportunity to read it, but also to serve as a frame of reference and put the review in context.
Synopsis of the book
The book is entitled ‘Country-Led Monitoring and Evaluation Systems, Better Evidence, Better Policies, Better Development Results’. It is authored by 21 seasoned experts from various backgrounds, including evaluation organisations and associations, universities, independent consultants and United Nations organisations such as UNICEF and the World Bank; it was edited by Marco Segone from UNICEF. It includes two chapters: (1) ‘Why country-led monitoring and evaluation systems?’ and (2) ‘Good practices in country-led monitoring and evaluation systems’. The book tries to bring together the vision, lessons learned and good practices from different stakeholders on how monitoring and evaluation systems can enhance evidence-based policy-making. In addition, it contains several sections that address various issues confronting country-led monitoring and evaluation systems from a theoretical standpoint to provide guidance on best practices.
Introduction
The purpose of this review is to provide an overview of the book, its key messages, its significance and its limitations and provide some ways forward. When I was asked to provide a commentary of the book, I was looking for an inspiring reason to accept the task. The subtitle of the book was very attractive: ‘Better evidence, better policies, and better development results’. Inspired by this, I decided to generate evidence to better support the commentary. Firstly, I asked a couple of colleagues at the UNICEF Regional Office if they had heard of the book or read it. Whilst the answer to the first part of my question was often ‘yes’, the answer to the second part was ‘no’. This further inspired me to gather more evidence. I therefore launched a quick survey within UNICEF in all seven regions amongst the planning, monitoring and evaluation officers. Some of the evidence generated, even though not robust enough, was used to support some of my views.
General views on the book
It is always challenging to comment on a book in which several authors address only the ‘M’ (Monitoring) of ‘M&E’ (Monitoring and Evaluation), some address only the ‘E’ (Evaluation) and still others cover both. Thus, I structured my views into three interlinked and essential parts: (1) my and other’s views, generally the contribution and positive aspects; (2) potential limitations; (3) critical views and conclusions with suggestions on the way forward.Every book is unique and obviously cannot address all the angles of the subject (s) it discusses. Country-led monitoring and evaluation is an important topic for developmental evaluations, donors and international NGOs. It carries several challenges in defining, conceptualising, designing and furthermore implementing a real-world, country-led evaluation. I, therefore, thank and applaud the authors and editors for pulling this book together. The authors and editor have made a very significant and relevant contribution to the world of M&E in development by addressing such a challenging subject, with examples. The book has been quite well read by UNICEF staff and perceived as a useful contribution on the topic. Among 41 staff surveyed in 32 countries and regional offices, 64% had heard of the book and 50% of those claimed to have read it. Almost everybody who read it (recognising the bias that the survey covered only UNICEF staff in seven regions) qualified the book as an important contribution to the advancement of monitoring and evaluation, and specifically, in fostering country adoption of monitoring and evaluation systems. Furthermore, 85% of those who read the book found it useful (35%) or very useful (50%); the rest rated it somewhat useful: ‘I found it very useful and a reference in supporting government to establish M&E association as well as support government M&E capacity development initiative. It is also very useful in reading what has worked in other countries’. (PM&E Officer, country level). However, during 10 informal interviews with managers (a few from within UNICEF and others outside UNICEF, including government staff), I found that even though some are aware of the book, only a tiny percentage have read any of it, and usually then only the introduction. A wider survey to a broader audience may be needed to better grasp the audience of this book, but limited research I conducted tends to point to the an audience of specialists in planning, monitoring and evaluation specialists, and few managers. What messages does the book carry? The following quotes from some of those who have read it address this question. The most important message from these quotes is that the key messages match the editor’s notes: ‘Emphasises the importance of evidence-based decision-making; evaluations and monitoring activities should not be donor driven but country led if they are to be utilised for improving policies in developing countries; presents information how to develop systems so that M&E is country led.’ (M&E Officer, country level). ‘Importance of the ownership of the M&E systems by the government for the achievement of better sustainable results.’ (M&E Officer, country level). ‘M&E systems should be owned and led by the countries and not pushed by donors: the development partners/donors should support strengthening existing mechanisms and systems and not push on establishing new ones.’ (Regional M&E chief).
Potential limitations
Despite the aforementioned positive views of the book, there are some limitations. The book does not distinguish ‘M’ from ‘M&E’ and ‘E’. While ‘M’ is much more understood and governments are much more involved and quite often in the driving seat, ‘E”’ is still misunderstood and still in the building stage in most countries in the developing world, and still donor driven. In addition, whilst the book’s title refers to ‘country-led’ monitoring and evaluation systems, the voices of government and country specialists with experience in implementing or setting up country-led monitoring and evaluation systems is quite silent, almost absent. Thus, the inclusion of the direct experience of government or donor staff who have had direct in-country living experience would have increased the reach and audience of the book, as well as the overall relevance. For instance, the Bosnia and Herzegovina example described in Segone’s write-up is excellent, and more of such examples by the direct implementers in country would have been of added value and significance. A quote from one reader emphasises this need for further real life examples: ‘There are many countries that have advanced in this. No need to reinvent the wheels, but refer to examples that can be possibly adapted to country specific needs’ (PM&E Chief, country level). An important question to be addressed is: What are the incentives for countries and donors to embark on country-led evaluations? An emphasis on the value of country-led evaluation could lead to some sort of incentives for donors and countries to foster such approach. Recent data from UNICEF meta-analysis in seven regions where it implements programmes showed that the quality of evaluation and its use tend to be high when it is country led and managed. Countries’ capacity to fully lead evaluation is one of the challenges described in the book as an impediment to country-led monitoring and evaluation systems. This, according to many views, represents an opportunity to assess and build in-country capacity to implement quality country-led M&E. However, the book does not fully describe how to address this lack of capacity. However, Keith Mackay’s text eloquently describes 13 steps from diagnosis to building capacity, including regularly evaluating M&E systems and creating incentives for data use for policy decision-making. An example, currently being tried by UNICEF, produces some food for thought and consists of building an evaluation team with internationally experienced consultants along with national consultants who can learn and be retained as national resources to foster country-led evaluations. Such an approach can build a cadre of national specialists to fill the capacity gap at the national level. The link between the different texts does not appear obvious, but on the other hand they all carry the same message: the value of country-led monitoring and evaluation systems, supported by best practices.
Conclusions and suggested ways forward
Robert Picciotto’s question (“Is the country the right unit of account?”) remains valid as we seek the right approach for regional monitoring and evaluation systems for regional and cross-border programmes. The book still provides very good steps toward operationalising the international declarations principles to which countries adhere, including but not limited to the Paris Declaration and Accra Declaration. Those who read the book would recommend it to others to read, 95% (n = 41) of UNICEF staff across seven regions stated that they would recommend it to others. A language barrier does exist, however, which makes the book not very accessible in non-English speaking countries, calling for translations into French, Spanish and Portuguese. Also, a good and concise executive summary, three to five pages, that brings all perspectives together would help readers to grasp the main ideas and then pick those areas to start with based on their needs. Even though the book provide a modality for picking and choosing the text to read, it is viewed as being too long and very limited in providing examples of real world experiences. When considering a second edition of the book it might be important to increase focus on guidelines, with specific examples of successful experiences with government and donors voices. How widely the book is disseminated and read could be assessed through a quick and wide survey and feed into the next updated edition of the book. How to raise incentives, not only at the government level but also amongst donors, remains an important question to address if development partners really want to address the issues of country-led monitoring and evaluation. This is compounded by the weak capacity in many developing countries, important issues that should make up an integrated package to foster country-led monitoring and evaluation systems. Jim Rugh’s idea of contextualising a plan for evaluation is very relevant as development programmes operate in different contexts and settings in terms of capacity, readiness, opportunities, human and financial resources and transition from emergency to development (very prevalent these days). Diagnosing these elements, per Keith Mackay, is certainly the first of the steps in designing and implementing country-led or region-led monitoring and evaluation. IDEAS’s survey on impact evaluation is a good example to diagnose the issues around region-led M&E, but should be broadened to cover more than just impact evaluation, which in itself is a challenge to be country led, especially in weak countries. Obviously, the book is quite pertinent and significant nowadays, as development partners seek the right balance to address coordination, country capacity in the area of evaluation, ownership and accountability in generating and using evidence to influence programme and policy decision-making.
Acknowledgement
Many thanks to Ibrahima Thierno Lo, Regional Knowlege Management Officer at UNICAEF WCARO, for his contribution in managing the survey to the potential audience of the book.
|