About the Author(s)


Fanie Cloete Email symbol
Department of Public Administration and Management, University of the Free State, South Africa

Citation


Cloete, F., 2018, ‘Policy indicator consistency and coherence in measuring public sector development programmes in South Africa’, African Evaluation Journal 6(2), a338. https://doi.org/10.4102/aej.v6i2.338

Original Research

Policy indicator consistency and coherence in measuring public sector development programmes in South Africa

Fanie Cloete

Received: 01 June 2018; Accepted: 20 Sept. 2018; Published: 14 Nov. 2018

Copyright: © 2018. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background: The South African (SA) government has over time developed separate policy indicator frameworks to measure governmental programme performances. They include a series of national development indicators to measure long-term societal transformation impacts, medium-term strategic framework indicators to measure the implementation of the National Development Plan, indicators to measure progress towards achieving the government’s 14 strategic programme outcomes and a separate set of environmental indicators. However, serious doubts exist about the efficiency of implementation management efforts to achieve optimal coherence among these frameworks to maximise the best good policy governance outcomes and make it possible to measure South Africa’s compliance with the United Nations’ Sustainable Development Goals (SDGs) Programme.

Objectives: The aim of the study was to review the coherence of implementation management of these different competing systems to maximise good governance outcomes in South Africa, and to propose strategies for improvement.

Method: The above indicator sets are summarised and critically assessed against the need to align them also with the envisaged SDG indicators.

Results: The contents and management of the current SA policy indicators can and should be significantly improved.

Conclusion: A more flexible and responsive integrated SA indicator set needs to be developed to comply better with the emerging international SDG system. This necessitates a longer term strategic approach to the SA system than the current short-term tactical approach.

Introduction

The South African (SA) government has developed over time separate policy indicator frameworks to measure governmental programme performances. They include a series of national development indicators to measure long-term societal transformation impacts, medium-term strategic framework indicators to measure the implementation of the National Development Plan (NDP), indicators to measure progress towards achieving the government’s 14 strategic programme outcomes and a separate set of environmental indicators. However, serious doubts exist about the efficiency of implementation management efforts to achieve optimal coherence among these frameworks to maximise the best good policy governance outcomes and make it possible to measure South Africa’s compliance with the United Nations’ Sustainable Development Goals (SDGs) Programme.

The purpose of the article is to analyse the coherence of these different competing indicator frameworks to maximise good governance outcomes in South Africa, and to propose strategies for improvement. The article commences with a brief conceptualisation of the need for policy consistency and coherence to achieve optimal policy goal achievement. It then summarises, critically analyses and assesses the main policy indicator systems that are currently used to measure and evaluate governmental performance in South Africa in general, to determine the extent of policy consistency and coherence that these systems exhibit. It concludes with a number of strategies to improve international Policy Coherence for Sustainable Development (PCSD) practices in South Africa.

Policy consistency and coherence as strategic policy design and implementation requirements

Measuring progress towards achieving policy goals is an important implementation strategy. The so-called 7-C Policy Implementation Protocol consolidated a number of good policy implementation practices (De Coning, Cloete & Burger 2018:chapter 7). To achieve success with policy implementation, the 7-C protocol requires the existence of appropriate policy content, context, commitment, capacity, coalitions, communication and coordination. It is beyond the scope of this paper to address all these dimensions of the 7-C policy implementation protocol. The focus will only be on the last requirement, namely the role of coherence in the effective coordination of policy implementation efforts (King et al. 2012; Picciotto 2005). Commentators are remarkably consistent in their views about policy coherence and its significance in achieving policy goals. The Organisation for Economic Cooperation and Development (OECD) defines policy coherence as:

the systematic promotion of mutually reinforcing policy actions across government departments and agencies creating synergies towards achieving the agreed objectives. Within national governments, policy coherence issues arise between different types of public policies, between different levels of government, between different stakeholders and at an international level. (Trinity Dublin College 2017)

The OECD also defines policy consistency elsewhere (2001:104) as ‘ensuring that individual policies are not internally contradictory, and avoiding policies that conflict with reaching for a given policy objective’. Coherence is defined as ‘a more “positive” reading in which different policy fields actively work together to achieve common overarching goals’ (Ashoff 2005:11). Policy coherence is also explained variously as ‘an achievement of a synergy between … policies’ (Gauttier 2004:23), and as ‘a “desirable plus” that “implies positive connections [and is] more about synergy and adding value”’ (Missiroli 2001:182).

Another author sees consistency in law as:

the absence of contradictions; coherence on the other hand refers to positive connection. Moreover, coherence in law is a matter of degree, whereas consistency is a static concept. (Tietje 1997:212)

Policy Coherence for Development (PCD), refers to:

the synergic and systematic support towards the achievement of development objectives within and across individual policies’, while ‘horizontal coherence refers to the coherence between a policy and other policies of the same political entity. Vertical coherence refers to the coherence between a policy at the EU level and the individual EU member states policies in the same sphere. (OECD 2012; Stross 2014:22–23)

The above perspectives are substantially the same. The OECD Policy Coherence Cycle is schematically represented in Figure 1.

FIGURE 1: The Organisation for Economic Cooperation and Development policy coherence cycle.

The main focus of the rest of this paper is on the question to what extent the content and application of current policy indicator systems in South Africa comply with the consistency and coherence requirements by the OECD for measuring progress towards achieving strategic government policy goals.

Measuring the achievement of development goals in South Africa

The measurement of development success in South Africa is a new endeavour by the SA government that has not yet fully come off the ground. The evidence for this statement is to be found in the existence of a number of parallel and isolated development measurement initiatives that have emerged in ad hoc ways over the last decade. These initiatives are briefly summarised and assessed in this section, to determine their respective degrees of compliance with the OECD models in this regard.

The South African development indicators

In 2003 the SA government identified a need to measure the general developmental progress made in the country since the establishment of the post-apartheid democracy in 1994, especially with key strategic governmental programmes. This resulted in the preparation by the Presidency of a 10 Year Review, which was eventually published in 2003 (PCAS 2003). No systematic indicators to measure progress were available for that purpose at the time and in 2006 government initiated a programme to develop systematic development indicators for future reviews. In 2008, the first set of comprehensive development indicators were published (Department of Performance Monitoring and Evaluation [DPME] 2008a). These indicators were based on the integrated indicator framework of the United Nations on Sustainable Development (UNCSD 2007). The UNCSD indicator framework identified four main sectors within which sustainable development results should be pursued. These are the social, economic, environmental and institutional sectors (UNCSD 2007). The SA Development Indicators initially comprised 74 indicators largely aimed at measuring socio-economic development in South Africa. It contained only one environmental indicator and a handful of peripheral governance indicators. It is further composed of mostly sectoral output indicators and a few ad hoc sectoral outcome indicators and very few integrated impact indicators.

This first (2008) development indicator set therefore did not constitute an integrated and synchronised indicator framework to measure progress towards achieving sustainable development in South Africa. It also did not attempt to synchronise the development indicator set with the activities that were at the time being undertaken in a parallel process under the guidance of the then Department of Environmental Affairs and Tourism (DEAT) to develop a National Framework for Sustainable Development (NFSD) for South Africa on the instruction of Cabinet. The NFSD exercise is summarised and assessed below, but for now it is important to state that the final draft of the NFSD deliberately omitted its provisional list of sustainable development indicators because of the Presidency’s Development Indicator Initiative. It therefore assumed that updated versions of the Development Indicators will also include more appropriate sustainable development ones. The latest version of the Development Indicators of the Department of Performance Monitoring and Evaluation (DPME 2014) is still, however, largely the same as its first version. It now contains 86 still largely output-based indicators, of which only one now attempts to measure sustainable development indirectly (a misnomer for trade with Africa), one on environmental sustainability (greenhouse gas emissions) and still only a few peripheral governance indicators (e.g. revenue collection, audits, corruption perceptions, budget transparency, public opinion on delivery of basic services) (DPME 2014).

The latest (2014) version of these Development Indicators is clustered in the following 10, largely UNCSD-based categories:

  • Economic growth and transformation (16)
  • Employment (4)
  • Poverty and inequality (7)
  • Household and community assets (6)
  • Health (9)
  • Education (10)
  • Social cohesion (9) (includes voter performance)
  • Safety and security (13)
  • International relations (5) (includes sustainable development and tourism)
  • Good governance (7) (mostly audits, corruption, citizen satisfaction and includes one on greenhouse gas).

As is evident from the above list of general development indicators, the overwhelming majority of them are skewed towards measuring socio-economic outputs and a few such outcomes in South Africa with no indication that the sustainability of those outputs and outcomes is measured. This indicator set has also not been updated or expanded since 2014, and is probably in the process of being replaced by a different and much more detailed competing set of medium-term strategic framework (MTSF) indicators that will also be assessed below. The SA Development Indicators clearly do not comply with either the OECD requirements for consistency and coherence or the integrated UNCSD framework for sustainable development indicators. It seems to be substantially defective as a best practices policy measurement instrument.

The state of the environment indicators

The state of the environment (SOE) indicator set is the earliest and most comprehensive initiative in the SA government to identify systematic environmental measuring instruments that are globally comparable. It is a specialised framework, focusing relatively exclusively on measuring environmental sustainability. It is also based on the UNCSD’s integrated model of sustainable development and its latest updated version also contains a number of peripheral socio-economic indicators to measure elements of human settlements such as land use, waste management and so on (Department of Education[DOE] 2016). One of the contributing reasons why the general SA Development Indicators do not contain more detailed environmental indicators is probably because of the existence of this relatively comprehensive set of such indicators that has been developed and used by the Department of Environmental Affairs (DEA) in its successive guises since the Rio summit of 1992.

The SOE indicator set is currently probably the most balanced and integrated framework available in South Africa so far, but obviously biased to the natural environment. Unfortunately, like the SA Development Indicators, it is not linked to any of the other frameworks. It is also used in isolation only for regular SOE reports. Despite a long legacy of systematic international cooperation, testing and refinement of this indicator set, the SOE indicators are, however, still isolated in their existence and application in the SA context, as is evident from the discussion so far.

The synchronisation, consistency and coherence imperatives that have been summarised above (OECD 2009) necessitate more effective integration of indicator systems in general and of the SOE indicators in particular. This should result in greater efficiency and effectiveness in policy implementation. An ideal opportunity to expand the SOE sustainability indicator set to also include the sustainability of other sectors presented itself in 2005 when work started on the NFSD under the leadership of the DOE in an earlier guise. However, this was not successful, as summarised below in more detail. It was also not incorporated in 2011 into the National Development Indicators of the Presidency. The SOE indicator set, however, still provides the basis for measuring the strategic environmental outcomes foreseen by the MTSF to implement South Africa’s NDP. This is another separate, silo indicator set which is addressed next.

The medium-term strategic framework indicators for the national development plan

The third isolated indicator system that exists parallel to the above-mentioned ones at national level in South Africa is the 5-year MTSF system that seems to be the dominant system now. It is supposed to measure the progress with implementing the government’s current 14 strategic outcomes within the framework of the NDP. The origins and history of this indicator system and its different but parallel tracks also followed a very convoluted and incremental evolutionary process that developed organically within the same organisation but in different units with different aims and implementation strategies without an overall coordinating mechanism to synchronise and align these various initiatives.

It started out as again an isolated attempt by the DPME in the SA Presidency to devise measuring instruments to assess governmental performance (DPME 2009) in the implementation of initially 12 but later 14 strategic outcomes that were envisioned by government in its quest to transform the country into a more equitable post-apartheid society (DPME 2010; PCAS 2003). At that time (2009) no coherent national vision existed, but a Government Wide Monitoring and Evaluation System (GWM&ES) (DPME 2007) was in the process of being set up to monitor and evaluate these transformation activities in government more systematically than happened in the 10 Year Review referred to earlier (PCAS 2003) and also in the subsequent 15 Year Review (DPME 2008b).

These 14 strategic outcomes are (DPME 2014–2019) as follows:

  • Basic education: Improved quality basic education
  • Health: A long and healthy life for all South Africans
  • Safety: All people in South Africa are and feel safe
  • Employment: Decent employment through inclusive economic growth
  • Skills: Skilled and capable workforce to support an inclusive growth path
  • Economic infrastructure: An efficient, competitive and responsive economic infrastructure network
  • Rural development: Vibrant, equitable, sustainable rural communities contributing towards food security for all
  • Integrated human settlements: Sustainable human settlements and improved quality of household life
  • Local government: Responsive, accountable, effective and efficient local government system
  • Environment: Protect and enhance our environmental assets and natural resources
  • Internal and external relations: Create a better South Africa, a better Africa and a better world
  • Public service: An efficient, effective and development oriented public service and an empowered, fair and inclusive citizenship
  • Social security: A comprehensive, responsive and sustainable social protection system
  • Cohesive society: A diverse, socially cohesive society with a common national identity.

The DPME developed detailed indicators in 2010 for each of the eventual 14 strategic outcomes of the SA government. Ministers and senior officials in line function departments, who were responsible for pursuing and achieving these outcomes, also signed delivery agreements to implement these indicators (DPME 2014–2019). However, in this development process there was no attempt to synchronise the national development indicators and the SOE indicators with the new strategic outcome indicator set.

The National Planning Commission (NPC) in the Presidency, which is a separate structure next to the DPME but reporting to the same minister in the Presidency, published a National Vision and NDP for South Africa in 2012 (NPC 2012). This comprehensive document of 444 pages spells out in detail what the government wants to achieve by 2030 in South Africa with its transformation goals. The plan is very detailed and technical but does not provide any envisaged measuring instruments to assess progress towards achieving those transformation goals. The National Vision and NDP are further not aligned with the 14 strategic outcomes that are managed by its sister unit, the DPME.

The vision of the NDP aims to eliminate poverty and to reduce inequality by 2030. It envisions a South Africa where everyone feels free yet bonded to others, where everyone embraces their full potential, a country where opportunity is determined not by birth, but by ability, education and hard work. Its specific aims are as follows:

  • uniting all South Africans around a common programme to achieve prosperity and equity
  • promoting active citizenry to strengthen development, democracy and accountability
  • bringing about faster economic growth, higher investment and greater labour absorption
  • focusing on key capabilities of people and the state
  • building a capable and developmental state
  • encouraging strong leadership throughout society to work together to solve problems.

These aims of the NDP are not aligned with either the 14 strategic outcomes of the SA government or with the goals of the NFSD and its implementation instrument, the National Strategy on Sustainable Development (NSSD), although it contains elements of the SOE goals relating to Outcome 10 on the environment. The NDP is in essence a comprehensive shopping list of nice things to have to improve South Africa. It is not a feasible business plan yet. By 2012, the NFSD/NSSD, National Development, SOE and strategic outcomes indicator systems were therefore running parallel but separate and isolated from one another at national government level in South Africa. Next to them a new NDP was established which was not linked to any of these systems and did not have its own indicator assessment framework. The 2012 NDP, now, however, constitutes the new strategic flagship development plan of government and now has to be synchronised with all the still existing legacy development programmes summarised above.

The current MTSF indicators are very uneven. Some are very detailed, while others are very superficial. They are also largely output-based with only a few that are outcome- and impact-based. Thorough revision, rationalisation and synchronisation of these indicators are necessary. Evolving government policies, however, make indicator stability and application even more difficult than they are under normal conditions. An example of this is the envisaged adaptation of the SA government’s economic policies, to focus on what is now increasingly referred to as Radical Economic Transformation (RET). Unfortunately, government is not clear about what it means by RET. Different conceptions of and approaches to RET exist within government and among different stakeholders in government. A more serious problem now is the non-alignment of the MTSF strategies and indicators with the government’s general development indicators, the SOE indicators and the NDP itself. This very brief summary of what now seems to be the most authoritative emerging indicator framework that is intended to measure the progress with achieving the strategic NDP vision for South Africa clearly indicates its lack of compliance with the OECD best practices. This is similar to the situation with the two other competing indicator frameworks assessed so far.

A fourth, largely abortive attempt to establish a comprehensive, integrated, evidence-informed and authoritative indicator framework to measure sustainable development in South Africa is assessed next.

The South African national framework for sustainable development and its national strategy for sustainable development

A draft National Framework for Sustainable Development was published in 2007 (NFSD 2007). This was the first attempt by the SA government to develop such a holistic policy framework for this country. It was necessitated by the resolutions taken during the Johannesburg Summit on Sustainable Development in 2002. Annex 2 of that draft framework contains a series of indicators to measure progress towards SDGs in different sectors in South Africa (Cloete 2015:65).

The indicator measurement approach that was followed in the draft NSDF report was, like the national development indicator system of the Presidency, primarily based on the UNCSD model referred to above, identifying the social, economic, environmental and institutional sectors within which sustainability would be pursued (UNCSD 2007). The proposed NFSD indicator framework was, however, also influenced by the sustainable development indicator framework that was developed by the Economic Commission for Latin America and the Carribean (ECLA) for application in Argentina (Argentina 2015; Martínez 2007). Dynamic indicators were conceived, indicating either positive or negative changes in policy outputs and outcomes over a specified period. The use of a combination of dynamic indicator ratios based on both stand-alone indicators and composite indices that reflect change over time is an important strategic departure from the traditional use of stand-alone, static, snapshot indicators that provide a picture of a given moment in time and which is outdated immediately after the measurement was taken (Rabie 2014). The different types of indicators used accommodate and reflect the complexity of the phenomena under investigation. This indicator framework has been summarised and assessed in more detail in Cloete (2015:65). The ECLA indicator model is probably the most sophisticated, evidence-informed indicator framework approach developed so far globally.

At the time, these provisional NFSD indicators still had to be synchronised with the national development indicators discussed above. They were developed by the National Indicator Initiative (NII) which formed part of the GWM&ES coordinated by the Presidency (GWM&ES 2005). Unfortunately, this never happened because of bureaucratic competition between the Presidency and the then DEAT (the lead department of the NFSD). In the end, the final approved NFSD did not contain any additional sustainability indicators (NFSD 2008). The NFSD initiative was developed under the guidance of the then DEAT, approved by the National Cabinet and published in 2008. This policy programme is still coordinated and managed by the successor to the DEAT, now called the DEA, who is also responsible for the management of the SOE indicator set. Since the development of the NFSD started in 2005 and after the publication of the final text of the NFSD and its implementation strategy (NSSD1 2011), the DEA, however, found it extremely difficult to implement the programme. The main reasons for this uphill battle include the following.

Firstly, the required sustainability mindset or culture of sustainable development in South Africa, which is imperative for success, is still largely rhetorical at most governmental levels. This is illustrated by the still overwhelmingly large number of output indicators that are used in the MTSF/NDP indicator set, despite the rhetoric of the outcomes approach that is supposed to go beyond outputs and to focus on outcomes. As is also evident from the general development indicators of the DPME, government prioritises short- to medium-term progress with socio-economic development and poverty alleviation in the country above all other potentially strategic longer term policy goals like sustainability of governmental programmes. This explains the predominance of mostly socio-economic output indicators and the significant absence of environmental and institutional outcome indicators in the general development indicator set of the DPME (2014). J.T. Radebe, minister in the Presidency, also illustrates this focus as follows in his foreword to the first SDG Baseline Report for South Africa:

The NDP presents a long-term strategy to increase employment and broaden opportunities through education, vocational training and work experience, public employment programmes, health and nutrition, public transport and access to information. While there are ‘quick wins’ to be had in each of these areas, the strategies will take time to have a large-scale effect on poverty. (Statistical Services of South Africa [StatsSA] 2017:Foreword)

Secondly, the DEAT and its successor, the DEA, are both relatively junior departments with small budgets compared to more prominent and senior government departments, such as Education, Wealth, Social Development and so on, that are led and managed by more influential political office bearers than the less influential departments. Successive ministers and senior officials in the DEAT/DEA found in the past and are still finding today that it is very difficult to persuade their ministerial and departmental colleagues in more influential and powerful departments to comply with the NFSD’s more gradual approach to resource use. The disagreement between competing advocates of short- and longer term benefit perspectives in government obstructs general consensus about the need to take a more strategic longer term perspective on development in South Africa to promote the sustainability of governmental programmes. This was illustrated during the development of the draft NFSD in 2007. A NFSD drafting delegation interviewed a senior economic planner in the Presidency about potential conflicts between the Accelerated Shared Growth Initiative for South Africa (ASGISA) developed in the Presidency and some principles of longer term sustainable development that were under consideration for inclusion in the NFSD which was in writing at the time. The response was that the interviewee could not see any conflict because the aims of the two programmes were different (Presidency 2006 [Union Buildings, Pretoria] pers. interv.). Continuing intra- and inter-departmental bureaucratic turf battles and empire-building competition among government agencies still preclude the reaching of consensus views about synchronising different, overlapping or competing government programmes to achieve higher degrees of policy consistency and coherence.

An important third constraint is weak national strategic political leadership in the Presidency where the synchronisation of government programmes has to be initiated and managed. This is evident in the absence of an enforceable regulatory framework in the form of compulsory legislative requirements to ensure policy consistency, coherence, efficiency and effectiveness. The combined effects, therefore, of a still largely short- to medium-term socio-economic poverty alleviation focus in government, accompanied by strong bureaucratic turf battles and weak compliance enforcement, have resulted in the current deadlock.

In 2011, the SA Cabinet approved a 3-year National Strategy for Sustainable Development and Action Plan (NSSD): 2011–2014 (https://www.environment.gov.za/node/21/edit?q=content/documents/strategic_docs/national_strategy_sustainable_development/) that provided a programme of action to promote five strategic priority areas of the NFSD during this period (NSSD1 2011). The NSSD contains a range of indicators for the five identified strategic priorities which are integrated planning and implementation systems, sustaining ecosystems and natural resource utilisation, a green economy, sustainable communities and responding to climate change. These indicators are, however, still very unsystematic and incomplete (Cloete 2015). Since 2014 no further updates or revisions to the NFSD and the NSSD have been made, and the status of this sustainable development framework is therefore uncertain. The NFSD/NSSD indicator development project was an attempt to create for the first time in South Africa a comprehensive, integrated indicator framework that would have complied fully with the OECD and UNCSD best practices for consistent and coherent policy measurement instruments. Internal bureaucratic turf battles and politics unfortunately scuttled this envisaged outcome.

At the time of writing, there was therefore no consistent, coherent, integrated set of policy measurement indicators available for use by the SA government. Although this situation is clearly untenable, the isolated use of the respective parallel and competing indicator systems that have been summarised and assessed above still seems to just muddle on in their respective silos since 2007 without anyone in government seemingly willing to commit themselves to improve this situation. In 2014, however, the UN adopted a highly authoritative set of SDG indicators. This presented a fresh opportunity to the SA government to improve this problematical situation by creating more policy consistency and coherence among its respective development programmes.

The Sustainable Development Goal indicators

In 2012, the UN established a full-fledged SDGs Programme, consisting of 17 strategic development goals that all its members states should aspire to achieve by 2030 and then be able to maintain in a durable manner (UN 2012; UNDP 2012; UNEP 2015; UNWG on SDGs 2014; Vandeweerd 2013). These goals are summarised in Table 1.

TABLE 1: Sustainable development goals.

The 17 SDGs can be regarded as idealistic in many developing countries that do not always have the basic operational implementation and management skills, experience and resources to pursue them as actively as is required to meet the envisaged outcomes by 2030. This operational constraint on achieving the desired goals is in all those cases also accompanied by a similar, higher order strategic management constraint to monitor and evaluate progress towards goal achievement. Nevertheless, all UN member countries, including South Africa, have accepted the challenge to pursue the achievement of these SDGs.

A comprehensive set of indicators to measure progress towards achieving the 17 relatively ambitious stated SDGs was also developed (Le Blanc 2014:4; UNWG on SDGs 2014:6). Bhamra (2015) emphasised the integrated nature of the sustainable development outcomes that are pursued, and stated very explicitly that:

social, economic, environmental and governance systems cannot be treated in isolation. For the systems to be concurrently aligned in the development paradigm, the first step is to develop a meta-metric framework that identifies indicators and their respective roles in the development processes. A clear comprehensive metric system that not just focuses on economic indicators but includes social, environmental and governance systems is a pre-requisite. (p. 1)

The latest version of 232 agreed-upon indicators to measure progress towards achieving the SDGs was published in 2017 (UNSC 2017). These indicators have been classified as follows (UNSDGs 2017):

  • Tier 1 (82 indicators): Indicators are conceptually clear, an internationally established methodology and standards are available, and data are regularly produced by countries for at least 50% of countries and of the population in every region where the indicator is relevant.
  • Tier 2 (61 indicators): Indicators are conceptually clear, an internationally established methodology and standards are available, but data are not regularly produced by countries.
  • Tier 3 (84 indicators): No internationally established methodology or standards are yet available for the indicators, but methodology/standards are being (or will be) developed or tested.
  • Multiple tiers indicators (5 indicators): Different components of the indicators are classified into different tiers.

In an earlier report published at the start of the SDG indicator development exercise, the UNSDSN (2015:3) also advised that ‘[a]ll SDG indicators need to be considered as an integrated package and must work in harmony with one another’. Christopoulos, Horvath and Kull (2012:305) conclude that sustainable development and its constituent elements and the need for integrating the social, economic and environmental aspects in development are widely accepted. The current global focus is now on improving the institutional framework. This view implies that a deliberate balance must be achieved among social, economic, environmental and institutional development to be durable or sustainable in the longer term (UNCSD 2007). Effective implementation of SDG programmes therefore requires not only basic project implementation and management capacities but also higher level sectorally integrated data monitoring, collection, analysis and assessment capacities in the form of appropriate evaluation systems and processes for these purposes. In this process, policy design and implementation consistency and coherence are important for success.

The latest OECD report on good policy coherence practices (OECD 2016:53) states that:

[t]he 2030 Agenda for Sustainable Development calls upon all countries to ‘enhance policy coherence for sustainable development’ (SDG target 17.14) as an integral part of the means of implementation. The report states further that ‘…[p]olicy coherence for development (PCD) has focused on avoiding or minimising the negative spill-over effects of various policies on the development prospects of developing countries. Policy coherence for sustainable development (PCSD) requires us to go one-step further, moving beyond a “do-no-harm” approach and towards a partnership approach based on “win-win” solutions. Importantly, PCSD will be fundamental for fostering synergies between economic, social and environmental policies in the implementation of the Sustainable Development Goals (SDGs), and take into account more systematically the effects of policies on the well-being of people living in other countries as well as of future generations’. (OECD 2016:19)

The OECD has also developed a PCSD framework consisting of eight building blocks, schematically represented in Figure 2.

FIGURE 2: Building blocks of policy coherence for sustainable development.

The 2017 PCSD framework builds on the 2009 PCD framework and applies its principles specifically to ensure that the potential of achieving SDGs in the longer term is maximised. Emphasis is, for this purpose, placed on the synchronisation of policy objective setting and prioritisation, with the effective coordination of policy implementation, monitoring and evaluation processes. Best practices to do this include the ensuring of logical coherence among the following main elements of the PCSD framework:

  • Accumulating evidence of committed leadership at the highest levels of government to mobilise all government resources behind strategic transformation programmes.
  • Integrating, synchronising and aligning related, supplementary programmes to avoid potential silo-effect contradictions and to mitigate potential negative implications of adopted programmes.
  • Taking a long-term transformation perspective that goes beyond single political electoral cycles and factors in longer term implications.
  • Institutionalising coordinating mechanisms to maximise success potential and to resolve programme conflicts effectively and efficiently, fully utilising public participation strategies.
  • Ensuring that appropriately institutionalised monitoring, evaluating and reporting processes and structures are in place to track implementation processes, results and impacts through clear target and milestone-setting and measuring levels of progress through appropriate indicators.

The OECD developed its PCSD model through practical experiences with the implementation of sustainable development practices in OECD countries. The PCSD framework is therefore not in principle substantially different from the PCD framework. It just contains more explicit details about the main requirements for longer term desired results. The coordinated, integrated approach that is followed both in the OECD’s PCD and PCSD models is as relevant for the SDG indicators as for the SA government’s national indicator systems discussed above because of the existence of a similar number of overlapping and competing international indicator frameworks to measure different aspects of sustainable development. They include inter alia:

Although many of these sustainable development indicator frameworks and systems overlap, they were all developed by different international agencies and interest groups for slightly different but overlapping and/or related purposes, without effective coordination. The result is that until recently, no consensus existed about which one of these systems could be said to be the best one to measure progress with achieving sustainable development outcomes and impacts. This situation is largely similar to the indicator development and application experience in South Africa, summarised earlier. In the case of the SDGs it changed when agreement was reached among UN member states to join forces behind its recent SDGs Programme. It is the latest attempt to try to rationalise the wide-ranging existing international approaches and to combine them into a single authoritative approach to sustainable poverty reduction across the globe. Successful poverty reduction also implies improvement in living conditions and therefore also in quality of life and hopefully in perceptions of well-being. This is what is needed in South Africa too.

The SDG indicator framework is the most authoritative international indicator system yet developed because it has been formally adopted by all UN member states. This system complies well with the consistency, coherence and integration imperatives required for optimal policy success.

Aligning South African policy indicator systems with the Sustainable Development Goals and Organisation for Economic Cooperation and Development policy coherence frameworks

Achieving SDGs necessitates buying into the longer term, more gradual, integrated resource use mindset underlying the UNCSD conceptualisation of what sustainable development is. As a member of the UN, South Africa has now also formally adopted the SDG Programme. This means that South Africa has to report regularly to the UN on its progress towards achieving the targets, milestones and eventually the strategic vision and goals of the global SDG Programme. Development programmes in South Africa that have not in the past accommodated SDGs, therefore, have to be re-aligned to include these revised goals in future because reports to the UN for this purpose have to apply the SDG indicators. This will inevitably have a very significant impact on the current application of the country’s different overlapping and competing indicator systems that are in any case seriously defective in terms of the UN and OECD requirements for integration, consistency and coherence of policy programme design and implementation. It is therefore clear that the authoritative nature of the SDG indicators will in principle necessitate a serious reconsideration and redesign of current indicator systems in the country to formally adopt the SDG approach and then redesign a single, authoritative, integrated and coherent indicator framework system for both domestic and international applications. This can only lead to improvement.

There seems to be a realisation that something needs to be done to break this persistent and detrimental silo approach to sustainability management in South Africa, but insufficient political will and initiative seem to exist to make the needed breakthrough. In 2016, only an inter-departmental working group under the lead agency of the Statistical Services of South Africa (StatsSA) started to try to work on some form of integrated compromise among the respective silo indicator systems that were summarised and assessed so far in this article. It started off as a working group that was truly representative of the main stakeholders in the exercise. These stakeholders included the DPME in the Presidency, DEA, National Treasury, the Department of Public Service and Administration – dealing with human resource issues, and representatives of various provinces, local authorities and other minor institutional stakeholders. The main task of the group was to draft an SDG indicator baseline report for South Africa. This was successfully completed and submitted to the UN in September 2017 (StatsSA 2017). The report succeeded in establishing baseline data for the SDG exercise in South Africa, but could not report on all the required SDG indicators. Table 2 contains a summary of the reported indicators.

TABLE 2: Indicators used in the 2017 sustainable development goals indicator baseline report for South Africa (SA).

The minister responsible for both the NPC that produced the NDP and the DPME that produced the GWM&ES, for example, stated in his foreword to the 2017 StatsSA Baseline SDG Indicator Report for South Africa that:

[o]ne of the key issues underpinning both the NDP and the SDGs is the necessity for an integrated approach to development that incorporates all sectors of society and fosters a mind-set and behavioural shift of ownership and agency…So, it falls upon us to reflect on whether we are being held back by insufficient collaboration, coordination and accountability on system-wide activities. There may often be good reasons why things are the way they are; but change for the better, sooner rather than later, we must. (StatsSA 2017:5)

This problem diagnosis from the responsible minister is very significant, but unfortunately the goals of cooperation and coherence that are stated are still largely rhetorical and little concrete progress towards achieving them has so far been made. Despite the acknowledgment of the head of StatsSA in the SDG Baseline Report that ‘the role of government departments, civil society, and international agencies that contributed towards the development of our Baseline Indicator Framework’ (StatsSA 2017:10), the reality was that towards the end of the drafting exercise of the report, it was largely an internal StatsSA effort because external stakeholders seemed to allegedly have lost interest in the conclusion of the report.

Conclusions

The need for balanced and integrated sustainable development outcomes across the globe is generally accepted as a prerequisite for sustainable human progress. The UN kick-started a major initiative already as far back as 1987 with the appointment of the Brundtland Commission (1987) to achieve this strategic global goal. This initiative is gaining strength, as the positive and negative global impacts of human activities across the world in social, economic, environmental and institutional sectors of society become clearer. Under the auspices of the UN, there is a clear international trend developing towards the measurement of progress towards achieving sustainable development in different sectoral dimensions. Sustainability assessment has to focus on all of these dimensions to achieve an accurate perspective on the durability of these envisaged policy interventions. An emerging good practice is further to assess sustainable development more accurately within a complexity and resilience paradigm.

The success of sustainable development outcomes depends on the effective synchronisation and reinforcement of balanced developmental programmes in the social, economic, environmental and institutional sectors of society. For this purpose, policy coherence and consistency is imperative to avoid policy programme contradictions and conflicts. The OECD model of policy coherence is an effective guideline to optimise the implementation of sustainable development programmes.

A comprehensive system of monitoring and evaluation of governmental programmes has been institutionalised since 2005 in South Africa. Over time it has slowly started to take root in a top-down way, driven and coordinated by the Presidency. This top-down model has also now been emulated in a number of other African countries and seems to be an effective strategy to fast-track the implementation of a more rigorous evaluation of the results of public sector interventions in those societies. Different parallel sets of policy programme indicators have also been developed in South Africa to measure different programmes. These indicator systems have been developed and still operate in parallel silos with little horizontal interaction and coordination. Current SA indicator systems are incomplete, defective in many cases, inconsistent and incoherent. They are, for example, in many cases fragmentary, contradictory, overlapping and not synchronised. Despite the SA government’s declared undertaking to move beyond the measurement of inputs, activities and outputs, which are currently still largely the foci of evaluations, to measuring the developmental outcomes and sustainable impacts of governmental programmes, this has not yet happened. Most SA indicator systems also conceptually confuse outputs, outcomes and impacts, and even where outcomes and impacts are supposed to be measured, output indicators are largely used for this purpose. These competing and overlapping indicator systems further have no alignment with global SDG indicators.

These weaknesses in the SA indictor systems mean that accurate measurements of outcomes and impacts, and especially of sustainable development achievements, are not currently possible. The implementation of these parallel systems further creates uncertainty, inefficiency, extra work and costs. Current competing national development, SOE, MTSF/NDP and NFSD/NSSD indicator systems in South Africa need to be significantly rationalised, revised and integrated into the newly adopted SDG indicators. This is necessary to achieve higher levels of accuracy, efficiency and effectiveness in measuring developmental outcomes and impacts, especially the sustainability of these developmental programmes, which is currently unfortunately still largely inadequate.

The draft indicators that have been developed for measuring the SA NFSD have the potential to establish a generic framework of sustainability indicators that is evidence-informed internationally, and would also be applicable with some customisation in other national contexts.

There is clearly a need for improved consistency, coherence, balanced and integrated measurement of sustainable development outcomes and impacts, not only for South Africa but also to compare this country’s progress with implementing the global SDGs. There are early indications of an awareness in government circles that this has to be done, but so far there is little evidence of progress towards this goal. The perception of urgency to rationalise still seems to be absent, probably because of strong vested interests, the complexity of the task and a significant lack of knowledge, skills and experience to do it. One can just trust that this situation will improve soon.

Acknowledgements

Competing interests

I declare that I have no financial or personal relationships that may have inappropriately influenced me in writing this paper.

References

Argentina, 2015, Sistema de Indicadores de Desarrollo Sostenible Argentina, Secretaría de Ambiente y Desarrollo Sustentable de la Nación, Buenos Aires.

Ashoff, G., 2005, ‘Enhancing policy coherence for development: Justification, recognition and approaches to achievement’, German Development Institute Studies 11, viewed 25 September 2018, from https://www.die-gdi.de/studies/article/enhancing-policy-coherence-for-development-justification-recognition-and-approaches-to-achievement/

Bhamra, A.S., 2015, ‘Resilience framework for measuring development: Brief for GSDR 2015’, Development Alternatives, viewed 25 September 2018, from http://www.devalt.org/images/L3_ProjectPdfs/Resilience_Framework_For_Measuring_Development.pdf?mid=6&sid=28

Brundtland, H., 1987, Our common future, Report of the World commission on environment and development, UN, New York.

Christopoulos, S., Horvath, B. & Kull, M., 2012, ‘Advancing the governance of cross-sectoral policies for sustainable development: A meta-governance perspective’, Public Administration and Development 32, 305–323, viewed 25 September 2018, from https://doi.org/10.1002/pad.1629

Cloete, F., 2015, ‘Measuring progress towards sustainable development in Africa’, African Journal of Public Affairs 8(3), 51–74.

De Coning, C., Cloete, F. & Burger, W., 2018, ‘Policy implementation’, in F. Cloete, C. de Coning, H. Wissink & B. Rabie, (eds.), Improving public policy for good governance, Revised chapter 7, 4th edn., pp. 195–233, JL van Schaik’s, Pretoria.

Department of Education (DOE), 2016, 2nd South Africa environmental outlook, Government Printer, Pretoria.

Department of Performance Monitoring and Evaluation (DPME), 2007, Policy framework for the government-wide monitoring and evaluation system, Presidency, Pretoria.

Department of Performance Monitoring and Evaluation (DPME), 2008a, Development indicators, Presidency, Pretoria.

Department of Performance Monitoring and Evaluation (DPME), 2008b, Fifteen year review, Presidency, Pretoria.

Department of Performance Monitoring and Evaluation (DPME), 2009, Improving government performance: Our approach, Presidency, Pretoria.

Department of Performance Monitoring and Evaluation (DPME), 2010, Guide to the outcome approach, Presidency, Pretoria.

Department of Performance Monitoring and Evaluation (DPME), 2014, Development indicators, Presidency, Pretoria.

Department of Performance Monitoring and Evaluation (DPME), 2014 – 2019, Medium term strategic framework: 2014–2019, Presidency, Government Printer, Pretoria, viewed 25 September 2018, from http://www.dpme.gov.za/keyfocusareas/outcomesSite/Pages/the-Outcomes-Delivery-Agreements.aspx

Gauttier, P., 2004, ‘Horizontal coherence and the external competences of the European Union’, European Law Journal 10(1), 23–41, viewed 25 September 2018, from https://doi.org/10.1111/j.1468-0386.2004.00201.x

Government-wide Monitoring & Evaluation System (GWM&ES), 2005, Government wide monitoring and evaluation system, The Presidency, Pretoria.

King, M., Keijzer, N., Spierings, E. & Matthews, A., 2012, Measuring policy coherence for development, European Centre for Development Policy Management (ECDPM), Maastricht.

Le Blanc, D., 2014, Towards integration at last? The sustainable development goals as a network of targets, Rio+20 Working Papers, 4, UNDESA, Division for Sustainable Development, New York.

Martínez, R.Q., 2007, Indicadores ambientales y de desarrollo sostenible: Avances y perspectivas para América Latina y el Caribe, ECLAC/CEPAL, Santiago.

Missiroli, A., 2001, ‘European security policy: The challenge of coherence’, European Foreign Affairs Review 6(2), 177–196, viewed 25 September 2018, from https://doi.org/10.1023/A:1011471421125

National Framework for Sustainable Development (NFSD), 2007, People – Planet – Prosperity: A strategic framework for sustainable development in South Africa, Draft for Comment, Department of Environmental Affairs and Tourism, Government Printers, Pretoria.

National Framework for Sustainable Development (NFSD), 2008, People – Planet – Prosperity: A Strategic Framework for Sustainable Development in South Africa, Department of Environmental Affairs and Tourism, Government Printers, Pretoria.

National Planning Commission (NPC), 2012, National Development Plan: Vision for 2030, National Planning Commission, Pretoria.

National Strategy for Sustainable Development and Action Plan (NSSD1), 2011, National Strategy for Sustainable Development and Action Plan: 2011–2014, Department of Environmental Affairs, Pretoria.

Organisation for Economic Cooperation and Development (OECD), 2001, The DAC guidelines – Poverty reduction, OECD, Paris.

Organisation for Economic Cooperation and Development (OECD), 2009, Building blocks for policy coherence for development, OECD, Paris, viewed 25 September 2018, from http://www.oecd.org/dataoecd/14/53/44704030.pdf

Organisation for Economic Cooperation and Development (OECD), 2012, Policy framework for policy coherence for development, Working Paper 1, OECD, Paris.

Organisation for Economic Cooperation and Development (OECD), 2016, Better policies for sustainable development, 2016: A new framework for policy coherence, OECD, Paris.

Organisation for Economic Cooperation and Development (OECD), 2017, Policy coherence for sustainable development 2017: Eradicating poverty and promoting prosperity, OECD, Paris.

Policy Coordination and Advisory Services (PCAS), 2003, Towards a ten year review: Synthesis report on implementation of government programmes, Policy Coordination and Advisory Services, The Presidency, Pretoria.

Picciotto, R., 2005, ‘The evaluation of policy coherence for development’, Evaluation 11(3), 311–330, viewed 25 September 2018, from https://doi.org/10.1177/1356389005058479

Prescott-Allen, R., 2001, The wellbeing of nations: A country-by-country index of quality of life and the environment, Island Press, Washington, DC.

Rabie, B., 2014, ‘Indicators for evidence-based measurement in evaluation’, in F. Cloete, B. Rabie & C. De Coning (eds.), Evaluation management in South Africa and Africa, Chapter 6, pp. 204–251, African Sun Media, Stellenbosch.

Statistics South Africa (StatsSA), 2017, Sustainable development goals: Baseline report 2017 – South Africa, Statistics South Africa, Pretoria.

Stross, S., 2014, ‘One goal, many paths – The promotion of policy coherence for development in EU policy formulation’, PhD, University of Cologne, Faculty of Management, Economics and Social Sciences, Cologne.

Tietje, C., 1997, ‘The concept of coherence in the treaty on European Union and the Common Foreign and Security Policy’, European Foreign Affairs Review 2(2), 211–233.

Trinity Dublin College, 2017, What is policy coherence?, viewed 25 September 2018, from https://www.tcd.ie/iiis/policycoherence/concept/what-is.php

United Nations (UN), 2012, Back to our common future: Sustainable development in the 21st century, SD21 Project Summary for policymakers, United Nations, New York, viewed 25 September 2018, from https://sustainabledevelopment.un.org/content/documents/UN-DESA_Back_Common_Future_En.pdf

United Nations Commission on Sustainable Development (UNCSD), 2007, Indicators of sustainable development: Guidelines and methodologies, 3rd edn., UN, New York.

United Nations Development Programme (UNDP), 2012, Triple wins for sustainable development, United Nations Development Programme, New York.

United Nations Environment Programme (UNEP), 2015, Time for global action, Our Planet, Magazine of the United Nations Environment Programme (UNEP), Geneva.

United Nations Statistical Commission (UNSC), 2017, Revised list of global sustainable development goal indicators, Report of the inter-agency and expert group on sustainable development goal indicators (E/CN.3/2017/2), Annex III, approved at the 48th session of the United Nations Statistical Commission held in March 2017, UN, New York

United Nations Sustainable Development Goals (UNSDGs), 2017, Tier classification for global SDG indicators, viewed 25 September 2018, from https://unstats.un.org/sdgs/iaeg-sdgs/tier-classification/

United Nations Sustainable Development Solutions Network (UNSDSN), 2015, Indicators and a monitoring framework for the sustainable development goals – Launching a data revolution for the SDGs, United Nations Sustainable Development Solutions Network, viewed 25 September 2018, from http://unsdsn.org/wp-content/uploads/2015/05/150612-FINAL-SDSN-Indicator-Report1.pdf

UNWG on SDGs, 2014, Open working group proposal for sustainable development goals, UN document A/68/970, viewed n.d., from http://undocs.org/A/68/970

Vandeweerd, V., 2013, Analytical framework for post-2015 SDGs, United Nations Development Programme, New York.


 

Crossref Citations

1. Evaluating policy coherence: A case study of peatland forests on the Kampar Peninsula landscape, Indonesia
Dwi Amalia Sari, Chris Margules, Han She Lim, Febrio Widyatmaka, Jeffrey Sayer, Allan Dale, Colin Macgregor
Land Use Policy  vol: 105  first page: 105396  year: 2021  
doi: 10.1016/j.landusepol.2021.105396