About the Author(s)


Reitumetse Zantsi Email symbol
Performance Monitoring Unit, South African Parliament, Cape Town, South Africa

Citation


Zantsi, R., 2020, ‘The evaluative role of legislatures in creating a responsive executive’, African Evaluation Journal 8(1), a432. https://doi.org/10.4102/aej.v8i1.432

Note: Special Collection: 9th AfrEA International Conference 2019.

Original Research

The evaluative role of legislatures in creating a responsive executive

Reitumetse Zantsi

Received: 19 Aug. 2019; Accepted: 25 Nov. 2019; Published: 26 Feb. 2020

Copyright: © 2020. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

The ability of legislatures to play a more effective oversight role is intricately linked to its ability to properly conceptualise and institutionalise its monitoring and evaluation (M&E) function. This article discusses the three main ways in which legislatures contribute to strengthening their oversight role over the executive, with a focus on the South African Parliament. The three ways in which Parliament creates a more responsive executive include: (1) utilising the results of evaluations conducted by departments, stimulating demand; (2) initiating Parliament-led evaluations, especially on transversal issues; and (3) the phenomena of legislature-focussed evaluations (self-assessments). The extent and nature of Parliament’s interaction will be explored including the internal arrangements for M&E. The unique positioning of legislatures lends itself to exploring the use of tried and tested methodologies in a new environment, while the oversight mandate provides a key opportunity to explore evaluative methodologies and tools, especially in assessing the link between executive action and citizen outcomes.

Conclusions include recommendations for collaboration between components within Parliament, internal systems and processes, including the interplay between the institution’s knowledge management system and strategic management processes, as well as the active building of an evaluative culture. It is only when evaluation becomes an integral part of how Parliament conceptualises its work that the legislature will begin to play an essential role in creating a responsive executive, thereby strengthening the national evaluation eco-system.

Keywords: Legislatures; Evaluation; Oversight; Accountability; Self-assessment; South Africa.

Introduction

The role of legislatures in overseeing the implementation of policies and programmes by the executive is an inherently evaluative one. Tracking whether budgets are spent appropriately, targets are met (monitoring) and assessing whether the lives of citizens have been changed for the better (evaluation) are at the core of what legislatures do (Goldman et al. 2012).

Monitoring and evaluation (M&E) theory and practice are the linchpins required to accelerate socio-economic development for the people who need it the most. It is only through iterative improvement to the implementation of policies and programmes that governments around the world can make the necessary developmental progress.

The establishment and maintenance of a national evaluation system (NES) in South Africa was therefore a key component of its developmental agenda. This article defines the NES as characterised by Goldman et al. (2018), ‘as a national system which guides how evaluations are selected, implemented and used’ in a particular country. While advanced economies have had well-developed NESs since the 1980s, African countries like Benin started developing them from 2007, while South Africa and Uganda began in 2011. Complementary to the system are national evaluation policies (NEPs), which determine what evaluation is, what should be evaluated and how, and define the use and communication of findings (Chirau, Waller & Blaser-Mapitsa 2018). The system is set up to facilitate the implementation of the policy, while the policy frames the purpose, responsibility and approach to evaluation.

The NES thus sought to align policies and frameworks with the mechanisms, structures and processes for collecting, storing, analysing and using M&E information for decision-making. Much of the literature on the development of the NES describes the efforts made by the executive branch of the state in instituting policies and practices that entrench M&E within government machinery. This is often referred to as the supply side of evaluation. Conversely, the role of legislatures is mentioned sparingly and mostly as a user on the ‘demand’ side of the system. The active participation of legislatures in evaluation remains a territory that has not been explored with the same vigour as that of the executive branch.

The aim of this article was to outline the ways in which legislatures, in particular the South African Parliament, can use evaluation to strengthen oversight and contribute to a more responsive executive. The article attempts to go beyond Parliament’s role in stimulating evaluation from the demand side and outlines the potential for contributing to the supply side, and particularly, the significance of self-assessments. Recommendations are made with regard to Parliament’s playing a more active role in evaluation.

Background

Monitoring and evaluation can be defined as a process of assessing performance against a set of pre-determined goals and targets, in order to manage current and future outputs, outcomes and impact for the purpose of improving performance and achieving results (United Nations Development Programme [UNDP] 2002). Monitoring and evaluation are two distinct but complimentary processes that involve the collection, analysis and reporting of data (monitoring), as well as the assessment of the relevance, efficiency, effectiveness, impact and sustainability of implementation (evaluation). More specifically, monitoring entails tracking performance or key outcomes over time, while evaluation is a systematic assessment that can attribute outcomes and impacts measured directly to implementation.

The policy formulation and practice of M&E in South Africa evolved between 1994 and 2011, and was driven by reforms introducing principles of policy management, planning and implementation within the public administration. The first reforms from 1995 to 2000 focussed on creating a foundation for results-based management, with a more concrete outcome-oriented policy agenda being reinforced through the 2007 Strategic Policy Framework to guide the government-wide M&E system. The institutionalisation of an outcomes approach took place through a number of initiatives from 2009, including the drafting of guidelines and plans, and the ultimate adoption of the National Development Plan (NDP) for South Africa in 2011 with long-term goals, indicators for tracking progress and targets by which to hold the state accountable (Engela & Ajam 2010).

Changes in planning, executing and evaluating the executive have had a profound impact on the work of Parliament, given its oversight responsibility. The identification of appropriate indicators1 and the setting of targets may sound trivial; however, their effect is significant. Indicators enable us to perceive differences, improvements or developments relating to a desired change (objective or result) in a particular context (Church & Rogers 2006). When monitoring and evaluating the progress made by a department or entity, or whether targets set are relevant, Members of Parliament can use the indicators to gauge whether goals and objectives have been met. Indicators can therefore be used to measure and monitor the achievement of expected results and establish the ‘success’ of a department towards reaching target aimed at achieving the country’s developmental goals. It was thus imperative that Parliament builds its own understanding of the outcomes approach and its related tools and techniques.

As a result, parallel to improvements by the executive, the South African Parliament evolved in similar ways, albeit later, and slower in progress.

The legislation governing the financial management of the legislative sector states (Republic of South Africa 2009):

[T]o regulate the financial management of Parliament and provincial legislatures in a manner consistent with its status in terms of the constitution; to ensure that all revenue, expenditure, assets and liabilities are managed efficiently, effectively and transparently; to provide for the responsibilities of persons entrusted with financial management in Parliament and provincial legislatures; to provide financial management norms and standards; and to provide for matter connected therewith. (p. 2)

The fourth democratic Parliament between 2009 and 2014 focussed on the implementation of Parliament’s Oversight and Accountability Model, which included the establishment of the Parliamentary Budget Office, whose role is to provide Members of Parliament with independent and objective analysis on matters related to the national budget. The fifth Parliament (between 2014 and 2019) sought to improve and strengthen the Oversight and Accountability Programme, with the introduction of more long-term planning, a focus on outcomes and the achievement of NDP goals, Africa Agenda 2063, the SADC Master Plan and the Sustainable Development Goals.

The role of Parliament as defined in the Constitution is to pass laws, ensure representation and public participation in policy-making, and oversee the executive and hold it accountable. With regard to oversight in particular, Parliament uses numerous mechanisms including convening house plenaries and holding various committee meetings for substantive deliberations on the performance of departments and their entities. The national Parliament and the provincial legislatures scrutinise the extent to which long-term policy priorities (outcomes and impacts) have been achieved through hearings, focussed intervention studies and reviews to examine the impact of transversal (particularly development-oriented) programmes. Figure 1 outlines Parliament’s role in the cyclical process from planning to audit:

FIGURE 1: Parliament’s role in the planning and budgeting process.

The national legislature has endeavoured to shift its focus from transactional-oriented approach (process and compliance) to a more transformational and outcome-oriented approach. This shift empowers the institution to align strategy to operations; integrate planning, budgeting and reporting, and ultimately strengthen M&E across Parliament; and encourage the use of evidence in decision-making.

While progress has been made, a key challenge for the legislative sector as a whole is the ability to assess or evaluate the executive against their constitutional obligations. In addition, this assessment should be done according to the commitments made to citizens through executive plans and policies, and the outcomes of the implementation of these plans and policies, while taking into account budgets and other constraints that may exist. There is growing citizen demand not just for service delivery but for Parliament to extract that accountability from the executive. This is significant given the reported decline in trust in public institutions globally, and in particular, Parliament. Recent evidence from an Afrobarometer study indicated that trust in the South African Parliament is at an all-time low, with 58% of the respondents indicating that they do not trust Parliament at all or just a little (Chingwete 2016). A results-based approach is thus central to refocussing the work of the executive, as well as that of the Parliament, towards the needs of the most vulnerable citizens. A ‘comprehensive results-based orientation to the public services’ requires a Parliament that explores the full extent of its evaluative function (Porter & Goldman 2013).

Parliament as an inherently evaluative institution

The basis for M&E in Parliament is primarily derived from the following legislative frameworks:

  • The South African Constitution (Act 108 of 1996), which defines the oversight mandate of Parliament and further requires Parliament to, amongst others, demonstrate transparency, accountability, effectiveness and efficiency, and promote good governance.
  • The Money Bills Amendment Procedure and Related Matters Act, 2009 (Act No. 9 of 2009), which provides for a procedure to amend Money Bills before Parliament and for norms and standards for Money Bills before provincial legislatures and related matters.
  • The Financial Management of Parliament and Provincial Legislatures Act (FMPPLA, Act No. 10 of 2009), which aligns the activities of Parliament with spending and therefore defines how institutional planning, performance improvement, and reporting should be done.
  • Oversight and Accountability Model for Parliament, which defines a framework that describes how Parliament conducts oversight. The model defines, amongst others: (1) the values and principles by which Parliament conducts oversight; (2) the mechanism or framework to conduct oversight; and (3) the processes and resources required for conducting oversight.
  • Policy imperatives for Parliament, developed by the executive authority, which defines the policy priorities for a specific term of Parliament.

There are three ways in which M&E manifests in the legislative environment. Parliaments can improve NES by demanding high-quality evaluations from departments in conducting their oversight function; they can lead evaluation efforts, especially for areas that are transversal in nature; and they can initiate self-assessments that not only assist in building an evaluative culture within legislatures but also give insights into the attainment of national goals as a result of the legislature’s actions. These three areas interact and reinforce the other, and in efforts to strengthen the NES, all the three areas need to be given due attention. The section below discusses all the three in more detail.

The demand for evaluation results

Demand for evaluation reports is mainly driven by the work of committees of Parliament. Ordinarily, committees scrutinise the plans and reports of departments, to determine whether they have spent public funds efficiently and if they have been effective in implementing their programmes. The oversight role of Parliament requires departments and other public entities to supply Parliament with information on the implementation of their policies and plans. However, there is evidence to suggest that this limited information is inadequate in truly determining the value of public funds spent (Erasmus & Van der Nest 2011). This monitoring of the executive is also done through plenary activities, mostly in the form of questions to ministers and deputy ministers, as well as the president and the deputy president. Evaluation results may also be requested by Parliament. Through their oversight function, Parliament can encourage the production of more and better-quality evaluations conducted by the executive. This, in turn, has the potential to improve the quality of oversight conducted by legislatures (Porter & Goldman 2013). One way of stimulating demand is through the inclusion of Members of Parliament on departmental evaluation steering committees, to boost the legitimacy of evaluation results and to improve the use of evaluation reports by parliamentarians. Internal efforts that encourage a proactive approach to evaluation should also be explored, as discussed in the next section.

The supply of evaluation reviews

In considering the work of a ministry and its entities, committees use a much broader array of information other than M&E reports, as depicted in Figure 2.

FIGURE 2: Information that Parliament can use to evaluate the executive.

While much of the literature acknowledges the contribution of Parliament to the demand side of the system, there is also a case to be made for Parliament-led evaluations. In instances where Parliament identifies an area needing evaluation, there is a constitutional case to be made within a parliamentary democracy like South Africa, for an independent legislature that is able to initiate and lead evaluation efforts. Given the asymmetry in resources (and information) available to the executive versus the legislature, it should become a central concern for Parliaments to not only demand better-quality evaluation from the executive but also initiate and conduct evaluations of their own (Abola 2014).

As an example of how this might work, South Africa’s legislative sector, through the Speakers’ Forum, in 2017 concluded the work of the high-level panel (HLP) on the Assessment of Key Legislation and the Acceleration of Fundamental Change. The panel focussed on assessing impact in the following areas: (1) poverty, unemployment and the equitable distribution of wealth; (2) land reform: restitution, redistribution and security of tenure; and (3) social cohesion and nation-building. Public hearings were held in all nine provinces to solicit inputs directly from the citizens, and expert reports were commissioned on selected areas. While useful, the HLP was a diagnostic evaluation, and Parliament must find space in its ordinary business to conduct targeted and even rapid evaluations of transversal issues across government departments and even clusters. This space does exist and can be utilised through engaging and facilitating collaboration between the internal components including the Office of the House Chairperson for Committees: Chair of the Committee of Chairpersons, the Parliamentary Budget Office and the Office on Institutions Supporting Democracy (OISD).

The role of self-assessment

Parliament conducting self-assessment is one way of pursuing Parliament-led evaluations, and thus, many of the institutional structures discussed above are key in tracking and monitoring the outcomes of Parliament itself. Self-assessments have the added benefit of not only holding Parliament to the same standards that it expects from the executive but also providing an opportunity for Parliament to learn as to which of its own programmes make the biggest difference in the attainment of developmental goals. Those evaluations focussed on Parliament’s outcomes, and impacts would in a sense take the form of meta-evaluations.

An earlier attempt at self-assessment was in the form of the Independent Panel Assessment set up in December 2006 by the then presiding officers of Parliament (South African Parliament 2009), which was appointed to conduct an assessment described in its terms of reference as:

[T]o inquire into, report and make recommendations regarding the extent to which Parliament is evolving to meet the expectations outlined in the Constitution and also to assess the experience and role of Parliament in promoting and entrenching democracy. (p. 4)

The assessment focussed specifically on the extent to which Parliament ensures that there is accountability, responsiveness and openness regarding the implementation of matters enshrined but not limited to Chapters 4 and 5 of the Constitution (South African Parliament 2017).

Currently, the Joint Standing Committee (JSC) on the financial management of Parliament is tasked with overseeing the management of the institution and can play a role in ensuring that Parliament adopts M&E tools and methodologies. In its effort to stay accountable to the JSC, parliamentary service has initiated the institutionalisation of M&E and has instituted a series of initiatives to improve performance. One of these initiatives has been the use of more structured strategic management processes, including the introduction of an Integrated Strategic Management Framework. In addition, there is focussed work in cascading the institution’s strategic plan to individuals using the balanced scorecard. Monitoring and evaluation finds expression in the latter part of the strategic management process and has thus provided an opportunity to enhance the use of M&E reports. The use of ‘business’-friendly theory and practice (strategic management and the balanced scorecard) thus aims to contribute to an environment more readily geared towards evidence-based decision-making (South African Parliament 2018).

Monitoring and evaluating its own outcomes builds an evaluative culture within Parliament and contributes to attitudes towards the usefulness (and limitations) of M&E results. The ability of Parliament to use its own data to make decisions can only have a positive impact on the use of evaluation evidence to hold the executive accountable.

Recommendations and conclusion

While evaluative activities can be found in different spaces within legislatures, M&E infrastructure in Parliament is still largely uncoordinated and disparate. The legislative sector is currently in the process of sense-making and building knowledge on how legislatures can be more proactive and responsive to the needs of South Africans. The legislative context is complex, and progress in institutionalising M&E lags behind that of the executive. In dealing with some of the issues identified above, the following recommendations are made:

  • Coordination and collaboration between different components within legislatures should be encouraged, such as the M&E Unit and core business components, with a view of creating a coherent M&E theory and practice that move the needle on both M&E demand and supply.
  • Internal systems and processes must be attended to, including the knowledge management and strategic management systems, as they have the biggest potential to improve the use of evidence in all areas of parliamentary work.
  • The active building of evaluative culture within legislatures through encouraging self-assessment efforts builds internal capacity to interact with evaluative information and helps sustain the NES.

As the bridge between citizens and the executive, the role of Parliament in strengthening evidence-based decision-making is critical for a sustainable evaluative culture. This article presents an approach to thinking about the role of legislatures in strengthening executive responsiveness. How Parliament conceptualises its own work has a profound impact on how it interacts with the executive. As M&E methodologies and tools are used more and more in the exercise of Parliament’s oversight mandate, as well as in self-assessment, the legislature can build a sustainable evaluative culture. This will only serve to produce better outcomes for Parliament, as well as create a lasting impact for the citizens who need it the most.

Acknowledgements

Competing interests

The author declares that she has no financial or personal relationships that may have inappropriately influenced her in writing this article.

Author’s contributions

R.Z. is the sole contributor to this research article.

Ethical considerations

This article followed all ethical standards for carrying out research without direct contact with human or animal subjects.

Funding information

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Data availability statement

Data sharing is not applicable to this article as no new data were created or analysed in this study.

Disclaimer

The views and opinions expressed in this article are those of the author and do not necessarily reflect the official policy or position of any affiliated agency of the author.

References

Abola, C., 2014, Does the Parliament of Uganda have a role to play in evaluation?, viewed 15 February 2019, from https://ecduganda.wordpress.com/2014/12/03/does-the-parliament-of-uganda-have-a-role-to-play-in-evaluation/.

Chingwete, A., 2016, In South Africa, citizens’ trust in president, political institutions drops sharply, Afrobarometer Dispatch No. 90, Institute for Justice Reconciliation, Cape Town.

Chirau, T., Waller, C. & Blaser-Mapitsa, C., 2018, The National Evaluation Policy landscape in Africa: A comparison, Policy Brief, Centre for Learning on Evaluation and Results-Anglophone Africa, University of the Witwatersrand, Johannesburg.

Church, C. & Rogers, M.M., 2006, Designing for results: Integrating monitoring and evaluation in conflict transformation programs, Search for Common Ground, Washington, DC.

Engela, R. & Ajam, T. 2010, Implementing a government-wide monitoring and evaluation system in South Africa, Evaluation Capacity Development Working Paper 21, World Bank, Washington, DC.

Erasmus, L.J. & Van der Nest, D.P., 2011, ‘Reporting on performance against pre-determined objectives: the case of South African national government departments’, African Journal of Public Affairs 4(3), 135–151.

Goldman, I., Byamugisha, A., Gounou, A., Smith, L.R., Ntakumba, S., Lubanga, T. et al., 2018, ‘The emergence of government evaluation systems in Africa: The case of Benin, Uganda and South Africa’, African Evaluation Journal 6(1), a253. https://doi.org/10.4102/aej.v6i1.253

Goldman, I., Engela, R., Akhalwaya, I., Gasa, N., Leon, B., Mohamed, H. et al., 2012, Establishing a national M&E system in South Africa. Poverty Reduction and Economic Management Network (PREM notes), Number 21, Special Series on ‘The nuts and bolts of M&E Systems’, World Bank, Washington, DC.

Mphunga, A., 2017, Integrated Strategic Planning Framework, s.n., s.l. (unpublished).

Porter, S. & Goldman, I., 2013, ‘A growing demand for monitoring and evaluation in Africa’, African Evaluation Journal 1(1), 25. https://doi.org/10.4102/aej.v1i1.25

Republic of South Africa, 2009, Financial Management of Parliament and Provincial Legislatures Act 10 2009, Government Printers, Pretoria.

South African Parliament, 2009, Report of the Independent Panel Assessment of Parliament, Parliament Printers, Cape Town.

South African Parliament, 2017, Report of the high-level panel on the assessment of key legislation and the acceleration of fundamental change, Parliament of the Republic of South Africa, Parliament Printers, Cape Town.

South African Parliament, 2018, Integrated strategic management framework, Parliament of the Republic of South Africa, Parliament Printers, Cape Town.

United Nations Development Programme (UNDP), 2002, Handbook on monitoring and evaluating for results, Evaluation Office, One United Nations Plaza, New York.

Footnote

1. An indicator is a quantitative or qualitative factor or variable that provides a simple and reliable means to reflect the changes connected to an intervention. An indicator ‘consists of information that signals change’.


 

Crossref Citations

1. From evidence to values-based decision making in African parliaments
Caitlin Blaser Mapitsa, Aisha Jore Ali, Linda Sibonile Khumalo
Evaluation Journal of Australasia  vol: 20  issue: 2  first page: 68  year: 2020  
doi: 10.1177/1035719X20918370