A proposed monitoring and evaluation curriculum based on a model that institutionalises monitoring and evaluation

At some point, the reason underlying absent or ineffective monitoring and evaluation (M&E) of development interventions in some African countries was lack of political will and resistance by influential bureaucrats and technocrats. This is partly because M&E provides information that is not desirable politically (Baradei, Abdelhamid & Wally 2014). However, as Porter and Goldman (2013) point out, in recent years, there has been a growing demand for evidence-based decisionmaking among politicians and bureaucrats – more so the former, as the continent becomes more democratic and citizens increasingly demand accountability from their ruling elite (Baradei et al. 2014).1 Therefore, Porter and Goldman (2013) point out that politicians in Benin, South Africa and Uganda have thrown their weight in support of M&E. To this list, we can add the Kenyan, Ghanaian, and Rwandese politicians. Inevitably, it seems the political weight is a little too much for the civil servants who have eventually taken to monitor and evaluate development interventions for compliance.


Introduction
At some point, the reason underlying absent or ineffective monitoring and evaluation (M&E) of development interventions in some African countries was lack of political will and resistance by influential bureaucrats and technocrats.This is partly because M&E provides information that is not desirable politically (Baradei, Abdelhamid & Wally 2014).However, as Porter and Goldman (2013) point out, in recent years, there has been a growing demand for evidence-based decisionmaking among politicians and bureaucrats -more so the former, as the continent becomes more democratic and citizens increasingly demand accountability from their ruling elite (Baradei et al. 2014). 1 Therefore, Porter and Goldman (2013) point out that politicians in Benin, South Africa and Uganda have thrown their weight in support of M&E.To this list, we can add the Kenyan, Ghanaian, and Rwandese politicians.Inevitably, it seems the political weight is a little too much for the civil servants who have eventually taken to monitor and evaluate development interventions for compliance.With missing details, the model may appear to be complex.Should that be the case, then we recommend that one reads the other article as well.
Fundamentally, Figure 1 links development (its components and processes), public policy (its components and processes) and the specified stages of the public policy cycle framework.First, development, which fundamentally entails 'change' in the short term, medium term and the long term (Summer & Tribe 2008) to a relatively desirable human aspiration (Slim 1995), has five components.These are cultural development (Burkey 1993;Gereffi & Fonda 1992; Jack-Akhigbe 2013), political development (Acemoglu & Robinson 2012;McFerson 1992), economic development (Sachs 2005), social development (Gray 2006;Roseland 2000;Watson 2012) and environmental development (Roseland 2000;Slim 1995).Second, as Summer and Tribe (2008)

Development
ImplementaƟon FIGURE 1: Illustrating the relationship between development (and its components and processes), public policy (and its components and processes) and the key stages of the public policy cycle framework.
as either assumptions or risks described in a results framework.Third, a development intervention and, therefore, M&E can be at policy level, programme level or project level (Kusek & Rist 2004).For us to understand these three levels of development interventions, we need to appreciate public policy before anything else.
Fourth, public policy -described by Jann and Wegrich (2007) as an applied social science discipline that uses multiple methods of inquiry and arguments to identify, formulate, implement and evaluate development interventions -points to the 'how' of the 'what' -with the 'how' being public policy and the 'what' being development.It also has five components, namely leadership (Simeon 1976), governance (Hill and Hupe 2014), political economy or simply macroeconomics (Simeon 1976), institutional arrangements or analysis and organisational arrangements or analysis (Porter & Goldman 2013). 2 Fifth, public policy processes include research (Geurts 2014), decision-making (Simon 1945;Zittoun 2009) and the public policy cycle (Lasswell 1956).
Lastly, Lasswell (1956) has proposed the seven stages of a policy cycle, that is, intelligence, recommendation, prescription, invocation, application, appraisal and termination.Thereafter, several authors such as Anderson (1975), Jenkins (1978), May and Wildavsky (1978) as well as Brewer and deLeon (1983) have proposed variations to this stage model.In these newer versions, the proposed technical stages include 'issue formation' or 'diagnosis', 'formulation', 'implementation' and 'evaluation' while the proposed political elements include 'policy agenda setting' and 'policy adoption'.To tally it to practice, we reduce these stages to four after incorporating 'agenda setting' as part of the diagnostic stage and 'policy adoption' as part of the formulation stage.This adaption matches with the system the South African Presidency is using after the Planning involves application of the theory of change, the results chain and framework as well as the logical framework to link the perceived 'impact' to the required 'outcomes' and 'outputs' to their corresponding 'activities' and 'inputs'.
The third stage (implementation) comprises management and monitoring of 'inputs' and 'activities' to produce 'outputs' meant to realise the intended 'outcomes' and consequently enact the intended 'impact'.This is why, much more directly, monitoring is a management tool for overseeing the use of inputs, undertaking activities and producing outputs.We should remember that these three parameters are not an end in themselves.Lastly, the evaluation stage implies stocktaking to assess if the produced outputs are leading to the outcomes meant to bring about the desired impact.This is summative evaluation bearing in mind that there are other forms of evaluation (which we discuss in the next section).In sum, the last two stages of a public policy cycle framework kick-start the description and discussion of M&E terminology.Lastly, note how the model links M&E terminology to the broader contextual terms much more systematically.

Monitoring and evaluation: Components, processes, established facts, issues and debates
Figure 2 illustrates that the components (structure and function) of M&E studies are monitoring, formative or design evaluation, process or implementation evaluation and summative evaluation.We can further divide summative evaluation into outcome and impact evaluation.Over and above these components, the five elements of the results chain -that is, impact, outcomes, outputs, activities and inputs -and those of the results framework -that is, indicators and the accompanying data sources, baseline values, target values, assumptions and risks -make up the fundamental element of M&E with varying importance depending on which of the five components is of interest or under focus.
Processes in M&E include continuous data collection and storage, data processing and analysis, as well as reporting and integrating M&E results in planning, budget allocation and implementation.More specifically, Bakewell, Adams and Pratt (2003) describe some fundamental steps in data collection and processing as well as information management.More generally, Kusek and Rist (2004) as well as Görgens and Kusek (2009) provide a detailed discussion on M&E systems that can help us put this function together.
There are a number of established facts in M&E.For example, Baradei et al. (2014)

Proposed institutionalised monitoring and evaluation curriculum
Overall, had it not been for the systems methodology described in Wotela ( 2016), we would probably not understand the logical relationship between development, public policy, leadership and governance presented in Figure 1.Therefore, the first module on the M&E curriculum should be 'systems thinking and methodology'.Apart from providing for the model, M&E is logical by its nature and, therefore, requires a module that should equip students with systems thinking skills.Such skills will cater for important functions in M&E such as formative evaluation as well as articulation of the theory of change, the results chain and the results framework.including those for the M&E function.This will also allow for an understanding of performance management and resultsbased management.We are aware that the model provides for leadership, governance and political economy as important components of public policy.However, these could be provided in the context of public policy rather than as standalone models.
Fourth, most participating students -who by their own right are seasoned development practitioners and public administrators -felt that most M&E tuition assumes that development interventions are M&E ready.However, this is not the case and to allow for the M&E function, one has to reorganise the intervention.Therefore, the fourth module should be 'programme and project: strategic planning'.This should cater for a detailed understanding of the planning process especially diagnostics and formulation of development interventions as articulated in Figure 1.Relatedly, the fifth module should be 'programme and project: financing and budgeting' which should articulate how to deal with the inputs, that is, budgeting, including performance budgeting and financing.The students felt that currently this is oversimplified, if it is included, making it impossible for them to monitor and evaluate the inputs element of development interventions.
Fifth, the students felt that skipping implementation and management or administration of monitoring leaves a void in their understanding of monitoring and more so in their knowledge of how to actually monitor a development intervention in the real world.To allow for a holistic understanding of monitoring the sixth module should be 'programme and project implementation' which should articulate operations management and monitoring in the public sector.The module should also cater for a detailed understanding of operationalising development interventions.More specifically, a detailed understanding of implementing development interventions as well as monitoring data management: collection and storage, processing and analysis, reporting and integration into management decision-making.As shown at the bottom of Figure 1, the emphasis in this module should be management and monitoring of inputs, activities, outputs and, to a limited extent, outcomes.
Sixth, there was a heated debate around (1) whether there is a notable difference between research and evaluation and (2) whether good researchers make good evaluators or good evaluators make good researchers.Although this debate was inconclusive, the students did not see any notable difference between research and evaluation other than that evaluation is more applied research and that good researchers make good evaluators but not vice versa.We, therefore, propose a seventh module that caters for both 'applied research and evaluation'.This module should provide qualitative and quantitative skills needed for the formative, process and summative evaluation that was discussed in Figure 2. The module should strike a balance between rigorous research approaches and relevant research even if research and evaluation could be different.Patton (2008), in Porter and Goldman (2013), has pointed out that evaluation is different from research only because it supports developmental efforts through provision of practical and specific answers to its challenges.
Seventh, one of the established facts in M&E is the process of continuous data collection and storage, data processing and analysis, as well as reporting and integrating M&E results in planning, budget allocation and implementation.Therefore, the eighth module should be 'data generation, analysis and management' to cater for data, information and knowledge generation, analysis and management.This should include an understanding of either basic or advanced statistics, if not both, which is an important ingredient in the conversion of data to information and knowledge.However, emphasis should be placed across the entire spectrum of data management, that is, data collection, storage (databases), processing, analysis and reporting.The students felt that currently there is too much emphasis on statistics, not knowing that the current challenge in M&E includes mere data collection.For example, how does an official in the national Department of Basic Education collect monitoring data in all schools in the country?
Lastly, students agreed that building a monitoring and evaluation system is not child's play and unfortunately when consultants are hired to undertake this task, it is not contextualised or does not respond to all the requirements of an organisation's need for M&E.This gets worse if the M&E personnel have no clue about what the important ingredients of a functional M&E system are.Therefore, the last module should be 'the monitoring and evaluation system and practice'.This module should put everything together and includes skills for developing and managing an implementation framework and its accompanying monitoring and evaluation systems for any development intervention.Contextualisation and application of the concepts should be the main focus of this module.
The first five modules should be extended to senior officials so that they understand the role of M&E in development interventions.Understanding development and its interventions and how it relates to M&E can instil a sense of how useful this function is to their primary function, which is crafting and delivering development through public policy.
The foot soldiers involved in the actual data collection should be exposed to modules 6, 7 and 8 so that they do not only understand their role but also understand how this links in with the actual requirement of data they collect and process.Of course, one can break the proposed modules into more than one module and possibly add modules not reflected here.We, however, reserve the detailed specification of each module to respective institutions and possibly a future discussion.
We have derived these modules systematically from the model derived elsewhere but re-presented here.Other than applying systems methodology and a detailed literature review, the proposed syllabus had an active participation of almost 300 postgraduate M&E students.Therefore, the approach, the model and the product are academically idealised but more so practically inspired.The proposed curriculum exemplifies what civil servants do and, therefore, has the potential to improve the quality of M&E expertise and hence the quality of evidence collected and the rigour of its analysis.Further, exposure to the proposed curriculum may help with institutionalising M&E and improve on its function or its institutional and organisational arrangements.These gains will improve the practice and, therefore, move towards reaping the rewards of this function rather than undertaking it for compliance.The proposed modules are not in any way meant to make those interested in M&E specialists in development, public policy or management, but is intended to assist them to measure development interventions much more effectively.

Conclusion
In sum, the assumption is that institutionalising M&E training will enhance the profession but more so the capacity to assess development interventions (the what) and public policy (the how).Jointly, this will provide the much-needed accountability and transparency in the use of public resources.
There are hanging questions about the training and professionalisation of M&E.For example, though indirectly, Baradei et al. (2014) However, this undesirable tendency to undertake M&E for compliance reasons is not limited to government departments and public institutions only.Instead, the political weight has also facilitated demand for training in M&E.This has compelled training institutions to jump onto the band wagon to teach M&E.Most often at the end of these training programmes, attendees do know what they have learnt but seem not to internalise this knowledge and worse still they cannot put their newly acquired knowledge into practice.This has led to what we term 'monitoring and evaluation training hopping', where participants move from one training institution or workshop to another in the hope that they will eventually decode this supposedly new technique and more importantly apply it to their work.Unfortunately, this desire has eluded them even after the next training and the next workshop and the next, because the constraint is not the conceptual ability of the participants.Instead, it is the training institutions that have packaged operations management and sometimes some features of strategic management or similar and called it 'monitoring'.Similarly, these institutions have packaged applied research skills and anything similar and called it 'evaluation'.Further, they have taught these modules without linking M&E to 'what it is' and 'what it should be http://www.aejonline.orgOpenAccessdoing' as well as 'how' and 'why' in the context of development interventions, often mentioning that M&E is a management tool for decision-making but omitting or failing to mention or detail management.Unfortunately, participants still blame themselves for not decoding M&E as well as failing to apply this function to their work after attending a training session, if not several.In this article, we introduce a model -derived elsewhere, using systems thinking -meant to institutionalise public management and M&E arrangements within the developmental and public policy landscape.We derived this model to contextualise M&E.Obviously, contextualising and, therefore, institutionalising M&E has capacity building implications.To cater for this, we briefly describe M&E: components, processes, established facts, as well as issues and debates, before proposing an institutionalised M&E curriculum.

The model: The relationship between development, public policy and the key stages of the public policy cycle framework
questions 1, 2, 5 and 6.Omission of questions 3 and 4 implies that one cannot link the various elements, components and processes in an academic field of study nor link such a field to another.This is crucial to M&E because it has various elements, components and processes.Elements include inputs, activities, outputs, outcomes, impact, as well as indicators, baseline values, target values, assumptions and risks while including the results chain and results framework as well as the different types of evaluation.Data collection, processing, analysis and reporting comprise some of the key elements of M&E.Therefore, to be comprehensive as well as critical, one has to discuss this field of study using the six questions.This is why we discuss M&E components, processes, established facts, as well as issues and debates before we propose the curriculum.
interest]?', (3) 'what are the components (structure and function) of [insert field of study of interest]?',(4) 'what are the processes in [insert field of study of interest]?',(5) 'what are the established facts in [insert field of study of interest]?',and (6) 'what are the key issues and debates in [insert field of study of interest]?'Note that in teaching academic fields of studies, most modules pursue Commission produced the Diagnostic Report in July 2011, which set part of the agenda of the African National Congress.
whose lives the intervention will change and, therefore, we should be able to deliver a relevant and sustainable development intervention.The second stage, formulation, 2.Though not as straightforward, but interrogation of public policy literature as well as personal correspondence with public policy specialists suggests that institutional arrangements or analysis and organisational arrangements or analysis are components of public policy.
Illustrating the relationship between monitoring and evaluation and its components and processes.
(Patel 2013)at M&E at project level provides narrow but specific operational and performance management insights and recommendations that are not transferable from one project to another unless the projects are really similar, if not identical.However, M&E at policy and programme level provides broader insights and recommendations.Second, to conduct quality M&E activities at any level, one needs accurate and high-quality data.Third and obviously, high-quality M&E activities are a product of teamwork of individuals with varied specialisations.Fourth, M&E is a costly exercise.Lastly, M&E activities are sensitive to the cultural and political context 3(Patel 2013).3.One should also familiarise oneself with the 'evaluation guidelines' that are packaged into four broad sections: utility, feasibility, propriety and accuracy.offer.Fifth, most M&E activities face data challenges such as accessibility, quality and accuracy.Further, despite gathering rich information, the results of studies that apply a qualitative research strategy are not generalisable.Those that therefore, unequal to the challenge.Relatedly, most institutions lack skills and expertise to analyse M&E data and report results.Tenth, they argue that M&E requires and also results in political and economic stability.It promotes transparency and accountability in the use of public resources.
question what a M&E practitioner should learn and know to what extent and how (short term versus long term or job-on training versus time-off training).Further, should M&E be a dedicated function or should it be part of other functions such as strategic planning, budgeting, implementing, managing and decision-making?Regardless, if a M&E practitioner is not involved in an organisation's core functions of formulating and implementing interventions, how will they incorporate empirical and experiential lessons that the M&E function is generating?If the M&E function is sitting outside the formulation and implementation functions, can personnel charged with the latter assess M&E reports to incorporate such lessons in their functions?Overall these questions are addressing the issue of streamlining and institutionalising the M&E function so that it realises its intended aim and objectives.In a future discussion, we propose how one can navigate through the M&E forest to become an established practitioner or professional since this is a career that individuals do not explicitly choose.