Policymaking in many instances does not follow proper diagnosis of a problem using evidence to justify why particular decisions have been taken. This article describes findings of a diagnostic review of existing challenges facing early childhood development (ECD) in South Africa. The review is part of the government's attempt to use information to drive policy in strategic areas. It is part of the role that the Presidency is seeking to play in ensuring government programmes are evaluated to ensure that money that is spent is spent on programmes that have an impact and that there is value for money. This article summarises the key findings of the diagnostic review that was conducted of policy, services and coordination. The results reveal that a broader definition of ECD programmes is needed to cover all aspects of children's development, growth and health, from conception to the foundation phase of schooling. Many elements of comprehensive ECD support and services are already in place and some are performing well. However, there are important gaps. Key ECD strategies for the future are identified. The diagnostic evaluation used a variety of methods, including desktop analysis, interviews and data analysis. Issues emerged around how to link the evaluation with other processes in the involved departments. A particular challenge was how to handle the transition to implementation of the findings, as responsibility shifted from the steering committee to the departments. The process worked well despite past challenges with coordination across government.

This article describes the development of the first evaluation in South Africa using elements of the National Evaluation System (NES) and a diagnostic review of early childhood development (ECD) (HSRC 2012), which piloted what have become the standard systems for the NES. The evaluation was carried out between October 2011 and June 2012. The article outlines how this evaluation was undertaken, the systems that were developed at each stage, the emerging results as well as the lessons around the evaluation system. The authors are from the steering committee for the evaluation.

The Department of Performance (now Planning) Monitoring and Evaluation (DPME) in South Africa's Presidency was created in 2010, with a mandate to use monitoring and evaluation (M& E) systems to improve service delivery amongst other systems. In 2011 there was a major push in DPME to start an evaluation system and a group of people from different departments led by the Deputy Minister of Performance M& E visited Mexico, Colombia and the Unites States to see how they were implementing evaluation. On their return the team brainstormed a National Evaluation Policy Framework, which was approved by Cabinet in November 2011. Whilst consultation was happening on the policy framework, work started immediately to get some evaluations moving, which could be used to develop and pilot the system. The group that visited Mexico and Colombia included the lead people in evaluation from the Departments of Social Development (DSD) and Basic Education (DBE); in discussion it seemed that early childhood development (ECD) was an area that could benefit from an evaluation. Specifically, a diagnostic was suggested to determine where the sector was at that point in time. In parallel, discussions had taken place with UNICEF around taking forward research relating to children. Discussions also took place with the Department of Human Settlements, but those evaluations only progressed later, during 2012. As ECD covers a number of departments, it was deemed important that DPME take the lead on the evaluation in order to overcome the tensions that may have existed between the different departments. So ECD was selected for the pilot and DPME chaired the steering committee. (This is discussed further in Amisi 2015, in this volume.)

The best way to give children the best start in life is often said to be through an integrated approach to ECD (e.g. NPC 2012). It has been shown that focusing on a single aspect of child development does not yield sustainable results, because children's development depends on other complementary inputs such as nutrition, opportunities for learning, healthcare and so on. The integrated approach includes programmes in health, nutrition, water and sanitation, early learning and psychosocial care. In May 2004 the South African government mandated the social sector cluster to develop an integrated plan for ECD. The political commitment to the expansion of ECD was demonstrated by increased budgetary provision, inclusion in high profile programmes such as the Expanded Public Works Programme (EPWP) for ECD and the President naming ECD as a government Apex priority in 2008 and 2009. In 2010 ECD was included in the delivery agreement (plan) for government's priority outcome 1 on the increased quality of basic education.

The ECD policy situation in South Africa was complex, with different departments having developed policies and legislation that speak to similar as well as sector-specific and age-specific issues on how to address children's needs. The three departments that had been core to the provision of ECD services were DSD, DBE and the Department of Health, with the Department of Women, Children and People with Disabilities (DWCPD) playing a monitoring role.

The National Integrated Plan for ECD (NIPECD) aimed to bring greater synergy and coordination to government programmes undertaken by various departments in the area of ECD. A need was expressed for an evaluation of ECD services as the NIPECD had been operating as a policy framework from 2005 to 2010 and had run its course. Plans had been made to review the policy. Secondly it was felt that South Africa needed to understand the present policy and programme environment as well as to identify the challenges experienced in the provisioning of ECD. The ECD diagnostic review was conceptualised to obtain answers on the above issues. This was to be a parallel process to the review of the NIPECD and one steering committee oversaw both processes.

In 2011 DSD embarked on a process to review the NIPECD, which was finalised in 2005. The NIP review came about as a result of government's commitment to improve provisioning in ECD, reviewing the commitments made by government in the NIPECD for birth to age four. The NIP review focused more on institutional issues including accountability, decision-making, leadership, coordination, planning and resource mobilisation and allocation. The NIPECD review investigated the gaps, constraints in the development, implementation and M& E of the plan and its impact, to a limited extent.

As the lead department for children from birth to school-going age, DSD launched an awareness campaign to promote ECD in local communities in collaboration with the ECD sector. A South African ECD conference was organised from 27–30 March 2012 in East London in partnership with various stakeholders in the field of ECD. The theme of the conference was ‘Building future leaders through ECD’. The conference aimed to share lessons and experiences in the field of ECD and was attended by 646 delegates including policymakers, donors, development agencies, faith-based organisations, civil society organisations and government departments. In preparation for the conference each province held ECD community dialogues where ECD practitioners had the opportunity to discuss the status of the ECD services in their respective provinces. The expected outcomes of the conference were improved knowledge of ECD programmes and services, a shared vision for the future of ECD in South Africa and the adoption and signing of a national declaration or resolution to inform government and the sector as a whole about the state of ECD and to provide recommendations for future policies and programmes.

Arising from the declaration, there was an expectation that an action plan would be developed. The ECD conference action plan included the resolutions of the SA national conference on ECD, recommendations from the NIP review as well as recommendations made by the ECD diagnostic review, which were presented at a plenary session at the conference. The SA conference report was subsequently discussed at a Social Services Cluster of Director Generals and taken to Cabinet by the Minister of Social Development in October 2012. The improvement plan arising after the diagnostic review was named the South African Integrated Plan of Action for ECD, and drew from the diagnostic review, NIPECD and the conference. This required some political sensitivity to deal with different agendas driving the different processes.

The NIPECD review, the diagnostic review and the conference have together played a major role in the reform processes that are being undertaken currently in ECD in the country.

As the evaluation progressed, at each stage the question arose as to what system should be developed to deal with that stage of the evaluation – in other words the ship was ‘being built as we sailed’. Progressively the following guidelines and systems were developed:

• Guidelines for terms of reference for evaluations.
• Standard service level agreements (SLAs) for evaluations.
• A guideline for inception phases of evaluations.
• Guidelines for peer reviews (which recommended a methodology and a content peer reviewer for each evaluation).

Other systems were developed and used for ECD and only after they had been tested were codified into guidelines and templates. These included:

• Evaluation project plan – including the preparatory work, the evaluation itself and the work required after the evaluation.
• Terms of reference for evaluation steering committees –including the norm that they would be chaired by the custodian department, with DPME providing the secretariat.
• A standard for short final evaluation reports – with a one-page policy summary, a five-page executive summary and a 25-page main report.
• Guidelines on management response to evaluation results.
• Guidelines on improvement plans to address evaluation findings – these were required to be produced within four months of approval of the evaluation report and involved a range of stakeholders.

The key partners in the diagnostic review were DSD, DBE, Health, DWCPD, DPME and UNICEF. Treasury also participated in some of the steering committee meetings. Because of its particular experience with ECD, the Human Sciences Research Council (HSRC) was selected as the service provider. They were contracted in late October with a methodology that was essentially focused on reviewing secondary data and interviewing a broad team of people with government and NGO experience.

The service provider team brought expertise in the areas such as implementation of early childhood interventions, data analysis, economic perspectives of early childhood interventions, reception year of schooling (Grade R), models of practice, child rights, policy, health and developmental difficulties. Given the limited time and the need for a set of high-level observations and recommendations, the work was done as follows:

1. Assembling and reading a large number of documents on ECD in South Africa, especially those commissioned during the last few years (more than 110 documents were consulted).
2. Bringing expertise in the various areas to bear on the subject and also conducting overviews of relevant literature.
3. Consulting websites and colleagues, including government officials, for specific information.
4. Hosting four panels with provincial stakeholders in Gauteng, Western Cape, KwaZulu-Natal and Free State to discuss the topics listed in the scope of work and our emerging perspective.
5. Meeting regularly with the interdepartmental steering committee to discuss progress and respond to queries and comments.

In addition to the short final report, 12 detailed background papers were developed, which informed the final report. The topics covered in the background papers were: (1) scientific evidence for the importance of early child development for human capacity, health and personal and social adjustment, (2) the role of the state: legal obligations to provide comprehensive early child development services, (3) An overview of the ECD policy framework in South Africa, (4) maternal and child health and nutrition, (5) parenting, (6) safe and affordable childcare, (7) opportunities for learning, (8) human resource development for ECD programmes and services for 0–4-year-olds, (9) Grade R, (10) government funding for ECD in South Africa, (11) cost and impact and (12) South African data.

The White Paper 5 on early childhood education and the Children's Act sketch a broad vision of comprehensive ECD services spanning early childhood, encompassing home-based, community-based and centre-based services across health, education, social protection and socioeconomic development. In practice, however, different sectors act largely in isolation from one another, without shared vision, goals and accountability, and there are significant gaps in services – particularly with respect to nutritional support for women and children, support for parenting and families and childcare support for very young children and children with special needs. The definition of ECD programmes in the Children's Act is at the moment limited to learning and support (early childcare and education). Moreover, it focuses on services provided in centres, which excludes much of the important work needed in the home to support parenting and young children's nutrition, learning and protection.

The diagnostic review and improvement plan recommends that a country strategy for ECD be developed to submit to Cabinet and that the Children's Act be revised. It suggests that a broader definition of ECD programmes than is currently in the Children's Act is needed to cover all aspects of children's development from conception to the foundation phase of schooling, notably covering the first 1000 days of life.

The evaluation suggests the strategy should include a common definition of ECD, agreed provisioning based on age, stage of development, socioeconomic circumstance and needs (including delivery services to reach poor and vulnerable children and promoting universal access), multidisciplinary and inter-sectoral teams with funding streams and mechanisms in line with outcomes and results, specific institutional arrangements of interdepartmental and inter-sectoral cooperation with clear protocols and mechanisms for information sharing. Norms and standards should be developed for differentiated services.

Achieving these goals also depends on new funding and resourcing strategies, especially for early childcare and education. The evaluation suggests the need for a decisive paradigm shift towards a rights-based ECD framework and accompanying funding model that recognises and is capable of realising the state's obligations to provide ECD services, especially for those living in poor families, rural areas and informal urban areas and for children with disabilities. Positive lessons from Grade R and birth registration point to the need to move towards a funding model that is government-driven and pro-equity.

The diagnostic review found that sectors act largely in isolation from one another, without shared vision, goals and accountability, and there are significant gaps in services – particularly with respect to nutritional support for women and children, support for parenting and families and childcare support for very young children and children with special needs.

As older rather than younger children are more likely to be enrolled in centres, the funding model leads to greater investment in children 3 to 4 years old than in children in the first 1000 days. Yet the first 1000 days are critically important to later health, achievement and well-being. Children aged 3 to 4 years who live in areas without registered centres, many of whom are poor and generally under-serviced, do not receive the subsidy support. Because most early childhood care and education (ECCE) services are private and not-for-profit (NPO) facilities, they depend on user fees, which the poorest families cannot afford. This leaves many areas of the country, and many families, without ECD services.

Services and resources need to be better balanced across the age range, with state assistance for the period from conception to age 2 (first 1000 days), more support for 3 to 4 year-olds, which needs to be provided more equitably, and support for all parents and families across the ECD age range.

The recommendations made include that ECD services should be comprehensive and should address the specific needs of different age groups from conception to age 9. This should include appropriate partnerships with civil society and the private sector. The elements recommended include:

• Family planning, healthy pregnancies and postnatal care to give children an optimal start in life from conception.
• Nutrition support for pregnant and breastfeeding women and young children through home-based, community-based and facility-based programmes. This should include support for food security.
• Birth registration, social security through the child support grant (CSG) and other instruments, subsidised housing and other state provisions for the poorest families.
• Supporting parenting through public education campaigns, as well as using the faith sector and traditional leadership, and care groups and companionship support through outreach programmes.
• Quality learning by young children encouraged at home and in groups, programmes and centres that focus on building enjoyment of learning, the confidence to learn from others, especially adults, and self-control and social respect so that children can participate in and contribute positively to social life.
• Preparation for formal schooling by enrolment and regular attendance in Grade R, with support for learning from parents and other adults in the home.

An equity-based approach was recommended to ensure that the state and its partners prioritise the provision of services and support to those children and families who most need them. The diagnostic review recommended that work should begin immediately in the poorest and most disadvantaged wards, with the most urgent step to develop a basic ‘ECD package of services’ to be rapidly expanded to reach vulnerable children. A core package of ECD messages should be developed for inclusion in training of home-based and community-based workers employed in different sectors who reach young children in the course of their work and for campaigns to promote effective social mobilisation and messaging on ECD issues. There also need to be campaigns for empowering parents using different platforms.

The improvement plan following the evaluation review recommended that costed implementation programmes be developed to provide comprehensive ECD services, including a basic package of services targeting the poor and vulnerable, providing both a universal and targeted comprehensive service and disaggregating the needs of different age groups.

In terms of coordination, successful ECD outcomes require a multidisciplinary approach. Limited coordination was shown to be a significant barrier to the advancement of ECD as a key national development priority. Due to the fact that the responsibility stretches over a number of key departments from a broad developmental perspective, the key ECD strategic objectives and the associated outcomes have been inadequately defined and resourced.

The diagnostic review suggested that effective inter-sectoral collaboration requires several pieces that are not yet in place. These include: a common ECD agenda and goals that are mainstreamed into relevant sectoral policies, programmes and budgets (this has not happened other than in DSD and DBE) and an integrated M& E process. It suggests that use must be made of existing facilities for the delivery of ECD services. These include primary healthcare centres, as well as mobile health services, ECCE centres and programmes, NPOs, one-stop centres, offices of traditional authorities, churches and other faith facilities, as well as municipal and provincial service points. ECD services require strong and coordinated inter-sectoral vision, commitment and action but the current coordination structures do not work adequately. High-level authorisation and legitimacy of a well-resourced central agency or mechanism is needed to drive forward key strategies for ECD.

This new road map must include a carefully considered M& E plan including a defined set of indicators that can track progress over time. Without grounded evidence it will remain difficult to leverage the kind of support and resources that are required for sustained quality ECD.

As indicated earlier, many of the systems for the NES were piloted in this evaluation. Some of the key learnings in the process were:

• Evaluations under the NEP can provide particular benefits where there are several departments involved in implementation and there is a challenge about effective coordination. DPME may need to chair the steering committees in some instances.
• There is also a challenge in moving from completion of the evaluation report, to the next stages of management response and improvement plan, where responsibility shifts back to the departments from the evaluation steering committee.
• The importance of briefing the respective ministers through the process, so that there are no political pressures to move in different directions.
• Initially the decision was that the evaluation report would only be presented to Cabinet with the improvement plan. In practice this delayed being able to present the evaluation findings and so the system was changed so that evaluation findings are presented to the relevant structures (clusters) and on to Cabinet committees soon after the report is finalised, not waiting for the production of the improvement plan.
• A sensitivity by departments to the findings of evaluations and a wariness that making the report public does not provide an opportunity to sensationalise critical findings to show government in a bad light.

DSD has moved forward in taking forward the improvement plan or action plan, including developing a revised ECD policy and specifically the Children's Act, with support from UNICEF. The revised ECD policy was presented to Cabinet in February 2015. An interdepartmental committee is driving the ECD work forward. A costing exercise has been undertaken on the implications of the comprehensive services, which implies that the full set will be too expensive and some strategic choices will have to be made. DSD is also splitting the ECD role from the broader responsibilities with regard to children.

The diagnostic review and improvement plan were well received both by the Heads of Education Committee (HEDCOM) as well as the Council of Education Ministers (CEM). An implementation plan is being finalised to ensure that the recommendations made will be actualised. It was agreed that the DBE should focus on the development of a curriculum as well as continue with the training of ECD practitioners. The draft National Curriculum Framework that will form part of the services that will be provided by the DBE would be published.

In addition to the specific results of the evaluation, the DBE has also been able to use a comprehensive evidence-based approach to the planning conducted for the integrated national programme of action. The balance between the recommendations made and the departmental mandates must be carefully navigated.

In addition, a focused evaluation was proposed to look at the reception year of schooling, Grade R. An impact evaluation was commissioned of this which is also reported on in this volume (Samuels et al. 2015).

To address some of the gaps identified in the ECD diagnostic review the Department of Health has incorporated strategies aimed at building capacity of women with young children both at household level and in crèches where children under 5 are cared for. The work of the ward-based health teams includes visits to households to provide health education and promotion around key maternal neonatal child health and nutrition topics (these include parenting skills and encouraging interaction and stimulation for children) and to facilitate access to routine and curative health services and other services where these are required. Another area the department has prioritised is the development of a menu guide for crèches to ensure provision of nutritious menus.

In addition, arising from this evaluation a separate evaluation was commissioned on nutrition programmes addressing children under 5. This highlighted key challenges around malnutrition, both under-nutrition and over-nutrition, and a major problem of stunting in children under 5. This report was completed in March 2014 and the report was taken to Cabinet in February 2015.

The DWCPD's mandate was to promote, coordinate and monitor the realisation of the rights of women, children and people with disabilities. DWCPD saw its role in the development of the M& E framework for the ECD sector as well as in advocating for improved political and administrative support for ECD. Experience in the development of indicators shows that the process of arriving at the indicators to be used is as important as the particular indicators that are monitored. Presently, the information systems of government departments are at different stages – and an ECD monitoring strategy must consider the particular strengths and weaknesses of each of these and its specific contribution to the full set of domains and indicators. This is a long process that requires a sustained coordinated M& E approach and process. DWCPD was promoting a comprehensive ECD M& E strategy within the broader child rights monitoring framework, including a regime of indicators that enable us to monitor specific outcomes relating to the general status of children during their early years of life as well as a set of indicators that track the performance of government's institutional and operational systems. DWCPD was split after the 2014 elections and the functions dealing with children and people with disabilities moved to DSD.

DPME has supported the costing model for the comprehensive package of costing services. It is playing a limited role in supporting taking forward the recommendations, including participating in the steering group for the revisions to the Children's Act.

The National Development Plan (NDP) has a significant section on ECD. The NDP echoes many of the issues raised in the diagnostic review. It talks of a nutrition programme for mothers and infants, universal access to ECD, addressing coordination weaknesses and norms and standards. It encourages innovation in service provision including home-based and community-based services. The state is responsible for ensuring that all vulnerable families receive a comprehensive package of ECD services. The NDP says that state funding for ECD should be widened and that current funding mechanisms are not adequate for the expansive ECD programme reflected in this plan. It also suggests that it is important to test and develop funding models that cover the comprehensive package of services (NDC 2012:300–301).

As indicated earlier, many of the systems now advocated for the NES were piloted with the ECD evaluation. These guidelines and templates are all now available on the DPME website, and some have been revised at least once. The system appears to work, but we now enter an interesting phase as the findings of the seven evaluations from the first NEP, all developed using these systems, start to come in. Some of these have challenging findings, testing above all the commitment in the NEPF that all evaluations should be made public. The issues arising and how these are dealt with over the next year will also be documented and will really test the systems that have been developed, how evaluation results are received by Cabinet and how far the recommendations get implemented.

A draft ECD policy was tabled at Cabinet in February 2015. The draft ECD policy introduces a number of new services to fill gaps identified in the range of services currently available, especially within the first 1000 days of a child's life, such as home visits to pregnant mothers by community health workers, pre-registration of income-eligible pregnant mothers to have access to the CSG, pre-Grade R, building of ECD centres in under-resourced and rural areas by government and the design of early learning that will support the cognitive and language development of young children from birth to school-going age. The pillars that support the policy have been agreed upon by the different departments and include:

• ECD as a public good: the government's recognition of ECD as a universal right and that ECD lays the foundation for the attainment of broader societal benefits requires the public provision of ECD services by government for all eligible young children in South Africa.
• Universal access: the government has committed to the attainment of universal access to ECD for all children in South Africa.
• Dual government-regulated model of public and private delivery of ECD programmes.

The authors declare that they have no financial or personal relationship(s) that may have inappropriately influenced them in writing this article.

M.D. (head of ECD, Department of Social Development) led the evaluation and follow-up. M.-L.S. (responsible for ECD, Department of Basic Education) participated in the evaluation process. R.S. (responsible at the time for M& E in the Department of Women, Children and People with Disabilities) participated in the evaluation process. T.L.M. (responsible for nutrition, Department of Health) participated in the evaluation process and a subsequent evaluation on nutrition. L.R. (DST-NRF Centre of Excellence in Human Development and Human Sciences Research Council, South Africa) led the evaluation team. T.W.M. (Department of Planning, Monitoring and Evaluation) chaired the evaluation steering committee and is the lead author of the article. I.G. (Department of Planning, Monitoring and Evaluation) supported the evaluation from DPME. T.B. (Department of Social Development) supported the evaluation as head of evaluation in DSD.