Every two years, the African Evaluation Association (AfrEA) organises an international conference, attracting evaluation practitioners and experts from Africa and across the globe. The conference acts as the foundation for promoting and advocating for AfrEA's ‘made in Africa’ approach, and allows knowledge sharing, collaboration and networking amongst a wide range of international organisations and individuals. The theme of the seventh AfrEA International Conference, which took place in Cameroon in March 2014, was ‘Evaluation for development: From analysis to impact’. The conference sought relevant answers as to how to place evaluative evidence at the heart of development policymaking and how to format it in a manner that will make sense to policymakers. With this understanding, the evaluation discipline will serve all levels of the development infrastructure: ensuring effectiveness, efficiency and sustainability of development spending.

AfrEA's international conferences provide opportunities to share insights into the challenges of an evolving development landscape and the emergence of new approaches and tools to tackle these challenges. The Rockefeller Foundation and The MasterCard Foundation are committed to supporting new innovations in development evaluation, particularly those that strengthen resilience and inclusive voice in development evaluation and decision-making. Both foundations collaborated to provide financial support to Pact and Oxfam to co-lead two conference strands which supported evaluation practitioners in sharing their work on emerging areas of evaluation practice in Africa: Technology-enabled monitoring and evaluation and Evaluating market-based solutions to development.

This summary presents an overview of technology and market-based solutions within the broader development field and, specifically, in relation to evaluation in Africa. It then presents a summary and reflection of the content and key themes presented in these two strands.

The development field has evolved rapidly in the past decade and the road ahead is likely to be more intensely shaped by transformative forces, including economic, environmental, geopolitical, societal and technological shifts than ever before (World Economic Forum 2012). Africa is no exception.

This changing context of development has profound implications for development evaluation and for the contribution evaluation can make to the quality of programming in the development field. The changing context is increasingly focused on the effectiveness of development interventions by national governments, international partners and, increasingly, by non-state actors: private investors, foundations and philanthropists. Pressing human needs are not being met by an official aid system that is short of resources whilst, by contrast, private investment for development is growing (Picciotto 2011). Two important facets of the changing context are the growth of technology and the development of an increasing number of market-based solutions in development.

In recent years, Africa has become a region of strong economic growth, reaching 4.7% in 2013 and forecast to reach 5.2% in 2014 (World Bank 2014a). Several African countries are amongst the world's fastest growing and potential opportunities for trade, investment and science and technology development are significant, especially in service areas such as software development and information and communication technology (ICT). In fact, ICT has become a driver of economic growth in Africa and has the potential to lead African economies further towards innovation, entrepreneurship and development. ICT infrastructure has grown exponentially in the region. The availability of Internet bandwidth increased 20-fold from 2008 to 2012, and now provides 629 million mobile subscriptions (69.3 per 100 inhabitants) and 172 million mobile broadband subscriptions (19 per 100) (International Telecommunication Union 2014). The continent is now focusing on the transformational potential of ICT in agriculture, education, health care, water and sanitation, the financial sector and government services. Kenya, for example, is the world leader in mobile banking. The M-Pesa mobile money platform, which has over 18 million users in Kenya, provides a wide range of services and has spawned a number of innovative social entrepreneurs and small businesses.

This ICT and mobile revolution calls for evaluators to adapt their monitoring and evaluation (M&E) practices. At the same time, ICT and mobile technology provide innovative ways to approach M&E globally, and on the African continent in particular. The technology-enabled M&E strand at the seventh AfrEA International Conference brought together an exciting and varied group of evaluators who presented their experiences and reflected on strengths and weaknesses of technology-enabled M&E. Presenters grappled with questions such as: how can M&E practitioners move beyond one-off mobile survey data collection to take advantage of ICT's transformational potential in routine monitoring, geographic information system (GIS) mapping and analysis, and innovative evaluation and data visualisation for improved timely decision-making?

Similarly, as development agencies place increasing emphasis on truly sustainable impact, focus is shifting toward interventions that include efficient and scalable mechanisms, and partnerships amongst civil society, the private sector and other key stakeholders. Over the last decade, private investment in low-income and middle-income countries has tripled and now represents half of the financial resources available to developing countries (European Commission 2014). Investment interest has grown in countries such as Nigeria, which has joined the ranks of middle-income countries and become the 26th largest economy in the world. Its economy is now estimated at a size almost 90% larger than previously indicated for last year (G.P. 2014). In addition, private sector participation in developing countries is increasing. For example, more than 110 countries worldwide have private sector investment in their infrastructure projects (World Bank 2014b). Prominent examples of inclusive businesses (business initiatives that also contribute to poverty reduction by including low income communities in their value chains) include: (1) M-Pesa, which has provided access to financial services to millions of people, particularly in Kenya, (2) Equity Bank, which provides banking and credit services to eight million customers in East Africa, and (3) SAB Miller, a British multinational beverage company, which sources its ingredients from approximately 50 000 smallholder farmers in six countries in East Africa.¹

For example, in the new field of impact investing,² research suggests that the global market potential of impact investing is on the order of $500 billion, with some upper estimates putting it at $1 trillion (Morgan 2010). This, coupled with new market-based solutions for development, far exceeds the total amount of development aid available.

Development evaluation must adjust to this reality by focusing on evaluating beyond (or in addition to) aid programmes, developing innovative approaches to evaluating market-based solutions for development, impact investing and other new innovative financing mechanisms for development and finding new ways to engage a wide range of stakeholders in providing ‘close to real time’ feedback to inform decisions and take advantage of the potential of new technology to enrich and transform evaluation practice.

The technology-enabled M&E field has evolved rapidly over the past decade. On the African continent, many examples attest to the value, challenges and risks of using technology as recent experiences in Kenya, Nigeria, Uganda and Cameroon highlighted during the conference.

Mobile technology for survey activities has been the initial starting point of using ICT in M&E. There are a number of different mobile data collection platforms available. Pact staff from Tanzania, Nigeria and Pact's global office led two learning sessions on the use of mobile technology for survey activities. The first learning session assisted participants in increasing their understanding of the use of mobile technology and the advantages and disadvantages of different applications and platforms (Kisyombe, Kudhlande & Akinmade 2014). The deeper-dive practical learning session gave participants hands-on experience in setting up a small survey, administering the survey and doing a preliminary analysis.

Mobile data collection clearly has a range of advantages over traditional surveying. The collection of data itself is faster and more accurate, quality is improved and there is the possibility of immediate feedback on data-collection challenges, monitoring of data collectors and viewing the results online for decision-making. Challenges include the potential lack of Internet connectivity, limited power availability, the initial learning curve, platform-specific technical limitations and the frustrations of initial survey and mobile phone setup. Opportunity International presented an interesting case study of its pilots of a number of mobile technology platforms for use in evaluating four country agriculture and micro-finance projects. In the end, the team decided to take a more traditional paper-based approach, citing technological, operational and financial constraints with the mobile technology platforms (Yamamoto 2014).

Alongside the more ‘traditional’ mobile survey data collection experiences, there are a variety of innovative uses of ICTs for M&E. Organisations are starting to use technology for routine project monitoring. For example, the Grameen Foundation presented a project in Uganda (Mugabi 2014) in which 1200 agriculture outreach workers (community knowledge workers) use smartphones for information sharing, real-time communication and survey and ongoing routine data collection.

Mobile technology and online mapping resources have brought previously highly specialised GIS activities within reach of a much wider programme and M&E user community. A good example is a project in Cameroon that uses participatory mapping methods and a GIS to enable youth to map and monitor water and sanitation challenges in municipalities and to work together with municipal counsellors to improve and expand water, sanitation and hygiene (WASH) services (Ambe 2014). Likewise, the availability of easy video recording capabilities on smartphones and tablets and online platforms, such as YouTube and Vimeo, provide opportunities to incorporate participatory video in evaluation activities. A peace-building project in Kenya recently used this approach to enable youth to express and document programme impact and to learn from their experiences (Muniz & Mukoya 2014). Finally, mobile technology can transform community loan and savings activities by improving recording and availability of data for decision-making, links to mobile money platforms and communication and cross-learning amongst saving groups through the use of social media. The Grameen Foundation developed Ledgerlink, a smartphone app that enables savings groups to record and monitor their transactions, and link them to mobile banking (Matovu 2014).

Finally there was a lively discussion around ICT-enabled M&E in the African context in a tech salon ³ session facilitated by Linda Raftree, an advisor to the Rockefeller Foundation Evaluation Office. Daisy Kisyombe (Pact Tanzania), Leah Farmer (VOTO-Mobile) and Josiah Mukoya (Mercy Corps Kenya) acted as lead discussants to get the conversation started. The discussion covered a broad range of topics, and was expertly captured by Linda Raftree on her blog (Raftree 2014).

Many ICT-enabled M&E activities still frequently focus on data collection efforts. Yet, there is much potential to use ICT across the whole spectrum of programme and M&E activities: big data analysis, social monitoring and feedback loops, mapping and geolocation, strengthening randomised controlled trials (RCTs) and sharing M&E information more effectively, to name a few (Raftree & Bamberger 2014). At the programme level, for example, mobile health (mHealth) projects are leading the way in utilising ICT to follow up with patients, provide SMS reminders, screen for diseases and provide accurate and up-to-date information to health care workers and patients.

Often there is a lack of integration with existing M&E systems or with the other mobile or ICT-enabled pilot activities. Few platforms allow for longitudinal data collection and integration of ICT in routine monitoring activities. Technology is not always the best solution to a given programme or M&E challenge. Technological tools should not be looking for a challenge to solve. They should rather be an additional versatile toolkit available to programme and M&E staff that can enrich and transform their practices, when appropriate.

To our knowledge, the use of ICT in M&E activities has not been systematically studied or evaluated. A quasi-experimental study of mobile versus traditional survey data collection could, for example, yield important information on differences in data quality, acceptability by survey staff and survey participants and the cost-effectiveness of the two approaches. Rigorous evaluations of ICT use in M&E practice will enrich and expand this developing field further.

During a series of dynamic panel discussions, the market-based solutions strand critically questioned issues in evaluating market-based solutions to development.

The strand's first session engaged attendees in a collective debate around the unique methodological challenges that evaluators face when assessing the impact of market-based solutions to development and the key questions that should guide every evaluator. Panelists shared their different approaches to this task, for example the Abdul Latif Jameel Poverty Action Lab's (J-PAL at MIT) use of the RCT methodology to assess agricultural and market interventions in Kenya (Bridgman 2014), a mixed-methods, difference-in-differences approach to evaluating the Horn of Africa Risk Transfer for Adaptation (HARITA) project in Ethiopia, designed to build resilience to drought amongst smallholder farmers (Madajewicz, Haile & Norton 2014), a mixed-methods evaluation of whether an Information, Education and Communication (IEC) campaign in Burkina Faso influenced enrolment in a community-based insurance scheme (Cofie 2014) and a cluster randomised evaluation to examine the impact of youth savings accounts on educational, health, economic, psychosocial and financial capability outcomes of youth and their families (Chowa 2014).

Participants in another of the strand's sessions discussed the challenges that implementers and evaluators face when assessing programmes in rural, resource-constrained environments, particularly the impact of efforts to address rural poverty that have focused on increasing access to financial services and markets. FHI360 shared its experiences evaluating a horticulture value chain in Liberia and its impact on children's well-being as well as at the household and farm levels (Rutherford 2014). A panelist from the Ministry of Finance, Planning and Economic Development in Uganda discussed an innovative evaluation paradigm that was used to assess the rural financial services programme intervention and how findings had led to improved project effectiveness and policy changes (Agabalinda & Kaspersen 2014).

Some of the strand's participants explored the critical question of whether agricultural and financial programmes succeed in their goal of increasing women's economic and social empowerment. Attendees collectively considered what happens when financially disadvantaged groups are economically empowered through inclusion in financial services or value chains. CARE International Rwanda and the Association of Microfinance Institutions in Rwanda (AMIR) explored CARE's Village Savings and Loans (VSL) approach, an integrated model that connects poor, rural women to the formal financial sector and allows them to set up and sustain enterprises, contributes to gender equality and has been able to influence Rwanda's national financial inclusion policy (Niyibizi & Rwema 2014). Ethiopia's Harbu Microfinance Institution described an operational and impact evaluation of its soy value chain financing project that assessed whether this project brought significant changes to women's livelihoods (Dinsa 2014). CARE International Mali presented lessons learned from the M&E system around its Pathways programme, designed to increase the productivity and empowerment of women farmers in agricultural systems (Fane 2014).

Another of the strand's sessions focused on impact investing, which is the newest area in market-based solutions and a topic of increasing interest. Impact investments are investments made into organisations and companies with the intention of generating social and environmental impact alongside a financial return. Whilst mostly focused on developed markets, emerging markets are fast increasing in importance and provide much-needed capital to support solutions to pressing challenges in areas as diverse as agriculture, health, technology and financial services. Guided by two presenters, from E.T. Jackson & Associates and Arabella Advisors (Harji & Mayer 2014), attendees used case studies to analyse this topic from the perspective of both an investor and investee. Key questions included: how can these investments be evaluated and what drives this evaluation work? What are the practical challenges that organisations face? And whose bottom line is it – clients, partners, private sector or organisations?

Two overarching challenges emerged from discussions:

• Balancing social and financial objectives: The rise of market-based solutions to development has meant that new stakeholders have come to the development table with their own, sometimes very different, objectives. More traditional aid programmes usually have a set of stakeholders focused on social impact goals, ensuring that programmes have impact for the intended clients, beneficiaries or target populations. Their approach and contribution to the objectives may differ, but there is generally agreement on the end goal of social impact, making the focus of measurement and assessment relatively clear. For-profit organisations have different objectives that are driven by financial benefit as well as social impact as an added benefit. Development evaluators must now balance M&E approaches, methods and tools to effectively measure both social and financial benefits. Impact can no longer be viewed solely in traditional terms and should be assessed in the context of both social and financial impact.

• With the different players now in the development arena, evaluators must grapple with the questions: whose evaluation is it and from what lens are we approaching our work? Evaluators must revisit how to measure success, as traditional approaches, methods and tools may not be relevant to private sector stakeholders. Assessing an initiative's bottom line will be increasingly important. Whereas an international non-governmental organisation (INGO) may view building resilience as evidence of success, an insurance company may need to see long-term profitability to ensure its future participation in an initiative. When evaluating market-based solutions to development, evaluators must be cognisant of these different lenses and the potential tensions between a programme's impact and financial bottom line. One of the most challenging areas will require making recommendations based on the data and evaluations and being able to weigh their effect, not only on impact but also the bottom line and vice versa. This is critical for evaluations to be convincing and for recommendations to be relevant for all stakeholders involved.

In addition to these overarching challenges, several common themes were salient across the strand's sessions. The first centred on how to capture the full effect of increasingly complex, multi-stakeholder programmes. These programmes often have several objectives: more traditional human development (income, livelihoods), empowerment (particularly for vulnerable populations) and the need to be financially viable and profitable (business case and returns). Critical to the task of assessing these programmes is the use of a mix of methods that can capture the changes related to various levels and areas of an intervention, some of which can be quite divergent. This goes beyond the usual reference to a ‘mixed-methods’ approach, which is often just a reflexive application of ‘qualitative and quantitative methods’. In addition, participants in the strand's methodology-focused session emphasised that considerations of methodological quality should be coupled with contextualisation and an acknowledgment of the limitations of various methodologies in certain contexts.

A second theme was the need for greater intentional connection between evaluation and policy. What is the application and utilisation of evaluation findings for policy formulation, influencing and advocacy? Although the assumption exists that higher quality data leads to greater application to policy issues, the cases presented show that this does not always happen. Considerable thought needs to be given to the policy question prior to the evaluation design and a plan put into place for utilising the results for specific objectives. The Youth Save initiative in Ghana (Chowa 2014), designed to increase youth savings and related positive outcomes amongst low-income young people in developing countries, has tried to intertwine programme with policy. Partnerships with financial institutions have opened the door for policy discussions with the Ghanaian Central Bank around regulatory issues for youth savings accounts. An assessment of a rural financial services programme in Uganda (Agabalinda & Kaspersen 2014) has also placed emphasis on policy by using an innovative evaluation paradigm in which knowledge management and policy dialogue play a central role.

Finally, the strand session that focused on impact investing (Harji & Mayer 2014) highlighted the potential for evaluators to join and shape the newest area in market-based solutions. The area of impact investing offers perhaps the starkest demonstration of having to look at different ‘bottom lines’ of impact on people's lives and financial return on investment. There is room for evaluators to draw from existing methodologies to inform the metrics and standards of impact investing, but they must be innovative in interpreting and applying findings.

As development interventions become more complex and additional forces drive change, there is a parallel need for the field of evaluation to innovate in order to move from analysis to impact. Momentum is increasing around a better understanding of how to evaluate market-based solutions to development and integrate technology into M&E and both areas are growing and rapidly changing. To truly understand impact in this changing context, we must move away from working in silos and creatively address methodological challenges.

Technology must cut across areas and inform the entire evaluation field. There is a need for information-sharing platforms that cut across projects, such as the AfrEA conference, where African practitioners present and discuss best practices and lessons learned. Targeted training will enable programme and M&E staff to systematically incorporate technology in their practices. Technology-enabled methods will benefit from methodological evaluation and validation.

Similarly, the innovations that arise from evaluating market-based solutions to development must also be shared and built upon. To remain current and useful, evaluation and monitoring approaches and mechanisms must transform to use innovative approaches to evaluate market-based development solutions and to incorporate the potential of ICT for better practice for decision-making. African M&E practitioners are well placed to lead the way in these developments.

The authors declare that they have no financial or personal relationship(s) that may have inappropriately influenced them in writing this article.

K.B. (Pact) and M.K. (Oxfam America) developed the initial idea, concept and framework for the review. C.A.M. (Oxfam America) and J.V. (Pact) wrote the manuscript. N.M. (The Rockefeller Foundation), J.D. (The MasterCard Foundation) and K.B. provided critical comments and guidance on the first draft. All authors reviewed and approved the final version.

Agabalinda, C. & Kaspersen, L., 2014, Linking realistic evaluation to knowledge management: The case of RFSP in Uganda, 7th AfrEA International Conference, Cameroon, March 2014.

Ambe, J., 2014, Digital mapping and governance project in Cameroon, 7th AfrEA International Conference, Cameroon, March 2014.

Bridgman, G., 2014, Using randomized evaluations, 7th AfrEA International Conference, Cameroon, March 2014.

Chowa, G., 2014, Ghana YouthSave experiment: An innovative collaboration, 7th AfrEA International Conference, Cameroon, March 2014.

Cofie, P., 2014, A mixed-methods evaluation of whether an IEC campaign influences enrollment in a community-based insurance scheme, 7th AfrEA International Conference, Cameroon, March 2014.

Dinsa, G., 2014, Impact evaluation of women inclusive innovative pro-poor financial services: The case of Harbu Microfinance Institution, Ethiopia, 7th AfrEA International Conference, Cameroon, March 2014.

European Commission, 2014 (13 May), Communication from the commission to the European parliament, the council, the European Economic and Social Committee and the Committee of the Regions, European Commission, Brussells, Belgium, viewed n.d., from http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX%3A52014DC0263&qid=1400681732387&from=EN

Fane, I., 2014, Participatory performance tracking in Pathways women in agriculture program, 7th AfrEA International Conference, Cameroon, March 2014.

G.P., 2014 (07 April), Nigeria's economy is bigger than everyone thought, The Economist, viewed n.d, from http://www.economist.com/blogs/baobab/2014/04/nigerias-economy-bigger-everyone-thought

Harji, K. & Mayer, W., 2014, Impact investments: Whose bottom line?, 7th AfrEA International Conference, Cameroon, March 2014.

International Telecommunication Union, 2014, Statistics, ITU, Geneva, Switzerland, viewed n.d., from http://www.itu.int/en/ITU-D/Statistics/Pages/stat/default.aspx

Kisyombe, D., Kudhlande, S. & Akinmade, O., 2014, Introduction to and overview of the use of mobile technology in monitoring and evaluation, 7th AfrEA International Conference, Cameroon, March 2014.

Madajewicz, M., Haile, A. & Norton, M., 2014, Managing risks to agricultural livelihoods: Impact evaluation of the HARITA program in Tigray, Ethiopia, 2009–2012, 7th AfrEA International Conference, Cameroon, March 2014.

Matovu, J., 2014, Technology tools for village savings and loan associations (VSLAS) in Uganda, Ledger Link, panel presentation, AfrEA Conference, March 2014.

Morgan, J.P., 2010, Impact investments: Emerging asset class, J.P. Morgan, New York, NY.

Mugabi, N., 2014, Using ICT enabled M&E systems for rural development: a case of the community knowledge workers (CKW) program in Uganda, 7th AfrEA International Conference, Cameroon, March 2014.

Muniz, S. & Mukoya J., 2014, The strengths and challenges of participatory video for Monitoring and Evaluating youth programming in Kenya, panel presentation, 7th AfrEA International Conference, Cameroon, March 2014.

Niyibizi, G. & Rwema, P., 2014, Growing together for significant results: The development of CARE's VSLA model, 7th AfrEA International Conference, Cameroon, March 2014.

Picciotto, R., 2011, Evaluating development philanthropy in a changing world, The Bellagio Initiative, Washington, DC.

Raftree, L., 2014 (17 March), ICTs and M&E at the African Evaluators’ Conference (Part 2), Wait … What?, viewed n.d., from http://lindaraftree.com/2014/03/14/icts-and-me-at-the-african-evaluators-conference-part-1/ and http://lindaraftree.com/2014/03/17/icts-and-me-at-the-african-evaluators-conference-part-2/

Raftree, L. & Bamberger, M., 2014, ICTs for M&E, a review of tools and applications, AfrEA Conference, March 2014.

Rutherford, D., 2014, A mixed-methods evaluation of an agriculture value chain intervention in Liberia, 7th AfrEA International Conference, Cameroon, March 2014.

United Nations Development Programme, 2013, Realizing Africa's wealth: Building inclusive businesses for shared prosperity, UNDP, New York, NY, viewed n.d., from http://www.africa.undp.org/content/dam/undp/library/corporate/Partnerships/PrivateSector/UNDPAFIMRealizingAfricasWealth.pdf

World Bank, 2014a, Africa's Pulse, Vol. 9, viewed n.d., from http://www.worldbank.org/content/dam/Worldbank/document/Africa/Report/Africas-Pulse-brochure_Vol9.pdf

World Bank, 2014b, Private Participation in Infrastructure Database, viewed n.d., from http://ppi.worldbank.org/explore/ppi_exploreSector.aspx?SectorID=2

World Economic Forum, 2012, Global Risks 2012, 7th edn., WEF, Geneva.

Yamamoto, G., 2014, Approaches, challenges and benefits in agriculture related evaluations, 7th AfrEA International Conference, Cameroon, March 2014.

1. Examples provided in UNDP (2013).

2. Examples of impact investing include: provision of below-market loans by a development finance institutions to small businesses, loan guarantees by charitable foundations in an affordable housing fund, which then can leverage additional commercial finance, purchase of the securitised debt of a microfinance loan portfolio by a commercial bank on commercial terms, which in turn helps the microfinance institution expand its lending, and purchase by private investors of shares or units in a green energy fund that then invests in local renewable energy projects (wind, solar) in poor communities.

3. For more information on Technology Salon, see http://technologysalon.org/.