Established in 2005, the South African Monitoring and Evaluation Association (SAMEA) is celebrating its tenth anniversary as a voluntary organisation for professional evaluation (VOPE). This VOPE occupies a special space in South African society as an independent and credible voice of monitoring and evaluation practitioners, drawing its membership from individuals working for government, academia, civil society, consultancies, donor agencies and private sector foundations, including students and the international community. As the biggest user of evaluations, government has become a natural partner for SAMEA since its first conferences organised in partnership with the Public Service Commission. The establishment of the Department of Performance (now Planning,) Monitoring and Evaluation (DPME) in the Presidency of South Africa in 2010 created new impetus for the institutionalisation of evaluation in government through the establishment of the National Evaluation System (NES). In 2012, DPME entered into a formal partnership with SAMEA to enhance monitoring and evaluation (M&E) capacity and continuous professional development. Working with the PSC and the DPME as well as other key partners like universities and development agencies, the role of SAMEA has been enhanced via the establishment of provincial associations, direct input to the development of government policy, evaluation standards, competencies and training – and co-hosting of high-profile conferences and workshops. The partnership between SAMEA and the government of South Africa is extremely useful for ensuring evaluation capacity development and institutionalisation of the practice in government and society. There have been tensions, especially as government's role in evaluation has strengthened. However, the significance of the partnership between SAMEA and the DPME has gained international interest and appreciation because of its structured nature, mutual respect, collaboration on capacity development and practical implementation of innovative projects.
Formally established in 2005 as a non-profit organisation in terms of Section 21 of the
This paper discusses the formative years of SAMEA as an association, the early partnership with the PSC in relation to hosting conferences, the more recent partnership with the DPME and joint delivery of projects such as debate and study regarding the options for professionalising evaluation in South Africa, establishment of provincial associations, evaluation awards, as well as implementation of international initiatives like the IOCE/EvalPartners Peer-to-Peer programme and plan for the 2015 International Year of Evaluation. The process has not always been smooth but a growing trust has been contributing to the growth in evaluation in South Africa.
The process of formally establishing SAMEA began in 2004 on the margins of the third African Evaluation Association (AfrEA) Conference, which was co-hosted in Cape Town by the South African PSC. This event drew over 400 delegates from 61 countries and represents a milestone in the development of monitoring and evaluation (M&E) on the continent and in South Africa in particular. As a result of the efforts initiated at the 2004 AfrEA Conference, SAMEA was formally established in 2005 as a non-profit (Section 21) association under South African law.
SAMEA's primary mandate continues to be the promotion and development of the practice of M&E in the public interest in South Africa. Whilst the contribution of the practice to the broader public good is the Association's foremost motivation, SAMEA also actively seeks to further the interests of its members, who are understood to be practitioners and users of evaluation.
In its early years, SAMEA received substantial support from the PSC, in an informal relationship that was based on shared purpose. Reporting directly to Parliament, the PSC is one of a number of constitutional entities exercising an oversight on the executive in the South African governance system. In terms of that mandate, the PSC is empowered to investigate, monitor and evaluate the organisation, administration and personnel practices of the public service. In fulfilling its role, the PSC pioneered the mainstreaming of evaluation in government, albeit from its formal position of independence from the executive. SAMEA and the PSC therefore shared the purpose of promoting evaluation as a practice in the public interest. In addition, PSC officials were pivotal in the establishment and early management of SAMEA and were represented on the elected Board. In 2007, the first SAMEA Board held the inaugural M&E conference with the theme ‘Evaluation in action’, followed by the second biennial conference in 2009, with the theme ‘Values in evaluation’. Both conferences benefitted from extensive PSC support. The rationale for continued collaboration seemed self-evident, and this important partnership continues.
In the year that SAMEA was launched, the South African government adopted the concept for a government-wide monitoring and evaluation system (GWMES), which was a significant move towards institutionalising M&E practice in the executive arm of the state. The GWMES Policy Framework was formally issued in 2007 (The Presidency
In its efforts to establish the NES, the DPME confronted a number of critical challenges, one of which was the dubious quality of government evaluations preceding the NEP. There was a need, therefore, to address the issues of quality, capacity and accountability. To do this successfully, the practitioner community had to be mobilised in a common cause. SAMEA was the formal representative of the community and explicitly shared these objectives. SAMEA also had credibility, good governance and a membership that was weighted towards practitioners in government and consultants serving government. SAMEA represented a credible partner that provided reach and access to practitioners independent of government structures and a ready vehicle to further shared objectives that would enhance the NES's effectiveness.
As such it was decided that the DPME and SAMEA would enter into a formal agreement that would preserve the independence of both organisations. This agreement was formalised in a Memorandum of Understanding (MOU) signed in February 2012, which set out how both organisations would cooperate in promoting M&E in South Africa. The MOU included the following:
Co-hosting and co-organising capacity building and learning activities.
Collaborating on setting standards and competencies for evaluators.
Working towards professionalising evaluation in South Africa.
Encouraging citizen participation and reporting.
A standing committee consisting of three DPME and three SAMEA members was established on 02 February 2012 in order to facilitate cooperation and collaboration. The standing committee meets after SAMEA Board meetings to discuss issues of common interest, based on the agenda approved by the Board, to ensure proper oversight on the work of the standing committee. Furthermore, the meetings of the standing committee are chaired by both SAMEA and DPME representatives on a rotational basis to promote collaboration and balance the partnership. Through this partnership, SAMEA has been able to provide direct input to the development of evaluation policy frameworks, guidelines, standards and competencies and M&E capacity development in the country, and the DPME has been able to promote wider awareness of the government evaluation system, as well as capacity development.
The next section provides a theoretical framework for discussing the lessons learnt from this partnership.
Literature on partnerships can be divided into three categories, namely normative, reactive and pragmatic analytical (Brinkerhoff
The ideal partnership is predicated on the normative perspective. There are two critical salient dimensions of ideal partnership, namely mutuality and organisational identity. The former encompasses the spirit of partnership principles, whereas the latter captures the rationale for selecting particular partners.
Brinkerhoff's partnership model.
Quadrant 1 (partnership) represents a situation in which mutuality and separate organisation identity are maximised. Quadrant 2 (contracting) represents a situation in which specific organisational characteristics and contributions are determined by one organisation, but sought in another based on organisational identity, to fulfil predetermined ends and means. Quadrant 3 (extension) represents a situation in which one organisation calls the shots and the other organisation has little identity and follows the dominant organisation's lead. Quadrant 4 (co-optation or gradual absorption) represents a situation in which a partner organisation compromises its identity by exchanging its services for the benefit of serving the dominant organisation, either consciously or unconsciously.
The next section outlines what the partnership has achieved, after which the paper analyses this experience according to the model above.
The partnership between SAMEA and the DPME as well as the PSC has resulted in significant achievements in relation to championing evaluation in the country and building capacity through a range of joint projects that are discussed below.
M&E practitioners in South Africa and throughout the world continue to debate various aspects and implications of the professionalisation of evaluation practice. Several regional evaluation associations are establishing thematic working groups (TWGs) on the professionalisation of evaluation. Amongst others, the proposed European Evaluation Society's (EES) TWG on Professionalisation of Evaluation is designed to identify and promote the collective actions needed within Europe to professionalise evaluation by expanding the supply of high-quality evaluation education and training; accelerating the harmonisation of ethical, quality and competency standards; increasing the autonomy of evaluation practice; and exploring the feasibility of designation and accreditation (EES
In South Africa, the establishment of SAMEA in 2005 marked the beginning of formal conversations amongst the M&E community about various measures that could be employed to continuously develop the evaluation profession. The SAMEA-DPME partnership through a standing committee is a notable development, as it draws from the capabilities of the two organisations and focuses on capacity building and continuous professional development of M&E in the country. The NES has several elements with regard to capacity development, including the development of competencies, standards, learning-by-doing support for evaluations and a suite of training courses. In 2014, the SAMEA Board established a portfolio committee on capacity building and professionalisation to advise the Board in relation to possible options that could be explored to professionalise evaluation. The DPME and SAMEA have come to a common understanding of the term professionalisation, defining it as ‘the process of continuous movement towards a greater measure of various professional characteristics and professionalism in the one who is practising evaluation’ (SAMEA
SAMEA, the DPME and the PSC are collaborating on several M&E capacity development initiatives. These include provincial workshops, which often coincide with SAMEA's annual general meetings (AGMs), setting up chapters, co-hosting the SAMEA biennial conferences and collaborating on the Evaluation Awards. Academic institutions and the Centres for Learning on Evaluations and Results (CLEAR) are involved in this capacity development programme. A summary of some of the key initiatives in this regard follows.
Historically, SAMEA and the PSC and later the DPME have co-hosted the SAMEA conferences, usually preceded by training workshops presented by local and international experts on contemporary topics. At the 2013 conference, the first following the establishment of the NES, the DPME was a major partner for the first time, and a stream on government evaluation was included.
Consultations on the development of government evaluation competencies and evaluation standards were conducted at national level with SAMEA and the provincial associations in 2011 and 2012, including a vibrant Western Cape learning network of M&E practitioners, comprising mostly SAMEA members and former Board members.
Both SAMEA and the DPME have recognised a need to acknowledge outstanding individuals and outstanding work in the field of evaluation in South Africa. Through identifying those who exemplify the very best in the field, they aim to honour both the evaluation practitioners and advance the emerging discipline of evaluation in the country. The first awards presentation took place at the SAMEA biennial conference in 2013, with awards in three categories, namely an Emerging Evaluator Award, a Best Government Evaluation Award and the Best Conference Paper.
The three partners have committed to continuously supporting the provincial associations in implementing a range of activities that would enhance evaluation capacity building and continuous professional development. As a result SAMEA, the DPME and the PSC have co-hosted evaluation workshops in collaboration with provincial M&E forums, and in the case of Eastern Cape Province this has become a provincial chapter of SAMEA. Most recently, SAMEA collaborated with the DPME and the PSC for the 2014 AGM to support the work of the Eastern Cape Chapter on M&E capacity building. The AGM was accompanied by a high-level opening session. In addition, there was a full-day session on SAMEA's flagship programme on emerging evaluators, half-day workshops on various topics and a closing symposium with higher education institutions on the conference theme.
A peer-to-peer (P2P) programme for VOPEs was initiated by EvalPartners
EvalPartners, in consultation with the evaluation offices of international organisations, including United Nations (UN) agencies and the World Bank's Independent Evaluation Group (IEG), the Organisation for Economic Co-operation and Development (OECD) Development Assistance Committee (DAC) and emerging countries, private foundations and other key stakeholders, convinced the UN General Assembly (GA) to designate 2015 the International Year of Evaluation in order to advocate and promote evaluation and evidence-based policy making at international, regional, national and local levels. Although the designation was led by EvalPartners and adopted during the NEC conference in Brazil in 2013, it is significant that it was officially adopted by the UN GA itself in December 2014.
In South Africa, the International Year of Evaluation has been led by SAMEA in collaboration with the DPME, PSC, CLEAR and other stakeholders. A steering committee and organising committee have been established to develop a coherent plan of events and products for the year, engage with international organisations promoting events during the year, assist with fundraising for these events and communicate with key partners.
The SAMEA-DPME partnership has elements of both the normative perspective and the pragmatic analytical model. In terms of the normative perspective, both parties promote values and principles of mutual influence, accountability and equality. The partnership also has a strong pragmatic analytical focus, especially as both parties view it as instrumental in reaching their respective objectives. The MOU is based on a general agreement, with specific agreements reached on a project-by-project basis, which characterises a pragmatic analytical perspective. Thus, the partnership envisioned by SAMEA and the DPME matches Brinkerhoff's ‘ideal partnership’ defined as ‘a dynamic relationship among diverse actors, based on mutually agreed objectives, pursued through a shared understanding of the rational division of labour based on the respective comparative advantage of each partner’ (Brinkerhoff
Partnership (Quadrant 1) represents a partnership in which mutuality and separate organisational identity are maximised. The SAMEA-DPME partnership fits into this category as discussed in this article. The SAMEA-DPME MOU clarifies the benefits for the partner organisations and partners’ roles under the agreement, and the standing committee is a collaborative mechanism for undertaking daily partnership activities.
Contracting (Quadrant 2) represents a situation in which specific organisational characteristics and contributions are determined by one organisation, but sought in another based on organisational identity, to fulfil predetermined needs. The SAMEA-DPME relationship is not determined by one organisation – both organisations need each other and their roles have been specified in the MOU.
Extension (Quadrant 3) is when one organisation is dominant and the other organisation has little identity and follows the dominant organisation's lead. As discussed in the partnership agreement above, SAMEA is a critical friend to the DPME and there have been disagreements between both organisations. Some fear that SAMEA is becoming dominated by government. This is something that has to be managed as government is becoming a bigger player in the evaluation field, both in setting standards and competencies and in funding evaluations.
Co-optation or gradual absorption (Quadrant 4) takes place when a partner organisation compromises its identity by exchanging its services for the benefit of serving the dominant organisation, either consciously or unconsciously. For the DPME, co-optation or gradual absorption of SAMEA would mean that there would be no independent peer reviewer to comment on DPME knowledge products, which could compromise the quality of these products.
There are fears of domination by government, and some stakeholders feel that government representatives should not serve as SAMEA Board members to ensure the independence of SAMEA. Despite positive spin-offs flowing from this partnership, some still argue that balancing representation of the DPME and the public service on the SAMEA Board is a threat to the independence of SAMEA, especially because the DPME accounts to Cabinet (i.e. politicians) and therefore cannot be independent. Similarly, some view the role of the SAMEA–DPME standing committee with suspicion, arguing that the DPME may dominate and SAMEA become an extension of the DPME, which could diminish SAMEA's identity. This situation has to be managed carefully to match the particular strengths each partner brings into the partnership, which will benefit evaluation in South Africa. At this stage, non-governmental and private sector involvement has reduced somewhat (e.g. in attendance and membership) as government has become more active. The Board and partners are aware of this and are trying to manage it carefully to ensure that the balance remains.
Although SAMEA and the DPME have worked together for some time, the relationship is still developing. The intention is to continue to build a strong collaborative partnership. This is dependent on developing suitable systems, e.g. the MOU and the standing committee, as well as trust between key individuals in the different organisations. Using Brinkerhoff's definition, it would appear to be a genuine partnership, although some worry about government becoming too dominant. The two organisations have renewed the MOU for the next three years (2015–2017) and plan to develop a three-year plan to concretise the MOU. This includes reflecting on several key projects, including the appropriate role and route for professionalising evaluation, and exploring a possibility of widening competencies and standards or adopting them as national standards and competencies.
The authors declare that they have no financial or personal relationships which may have inappropriately influenced them in writing this article.
T.B. (SAMEA) contributed to the introduction, sections on SAMEA's formative years, its partnership with the PSC, strengthening partnership with government through collaboration with the DPME and the conclusion. J.M. (DPME) contributed to sections on strengthening partnership with government through collaboration with the DPME, theoretical framework on partnership between government and non-profit organisations, key achievements of the partnership between government and SAMEA and lessons emerging from the partnership. S.N. (DPME) contributed to the abstract and the introduction. R.B. (SAMEA) contributed to sections on SAMEA's formative years, its partnership with the PSC, and strengthening partnership with government through collaboration with the DPME. V.N. (DPME) contributed to the section on lessons emerging from the partnership. M.L. (SAMEA) contributed to the section on key achievements of the partnership between government and SAMEA.
This is discussed more extensively in the paper by Ian Goldman
See the article on standards and quality assessment in this issue.
EvalPartners is an innovative partnership designed to enhance the capacities of civil society organisations (CSO) to influence policy makers, public opinion and other key stakeholders so that public policies are based on evidence, and incorporate considerations of equity and effectiveness. The objective of the initiative is to enhance the capacities of CSOs to engage in a strategic and meaningful manner in national evaluation processes, contributing to improved country-led evaluation systems and policies that are equity focused and gender equality responsive.