Original Research

Learning to learn: The experience of learning facilitation for grantees of Comic-Relief-funded projects

Dena Lomofsky, Jake Grout-Smith

Received: 13 Mar. 2020; Accepted: 27 July 2020; Published: 28 Oct. 2020

Copyright: © 2020. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

This article shares a learning facilitation process that Comic Relief, a UK-based funder, supported with its grantees. It is based on the case study of a learning cohort that Comic Relief established for one of its grants. The methodology for the case study is participatory action research. The case study demonstrates that by actively encouraging learning, grantees are empowered to adapt essential programmes to enhance programme effectiveness. Grant-makers need to consciously create the conditions for learning to encourage programme adaptation because organisations in international development (or local charities) have operated for many years under the dominant paradigm of results-based management. Results-based management tends to assert upward accountability based on low levels of trust and mechanistic processes for planning and reporting that attempt to control outcomes through forward planning and high-level decision-making. In this paradigm, organisations often end up proving a vision of success, rather than honestly learning from their implementation experience. However, a greater recognition of its unsuitability for working in highly complex contexts necessitates more learning by frontline workers and implementing organisations. This article shares our experience of how a carefully designed learning coordination and facilitation process encourages this kind of adaptation amongst grantees, generating lessons about design and implementation of projects that can be shared more broadly with others working with gang-affected girls and young women.

Keywords: Monitoring; Evaluation; Learning; Results-based management; Adaptive management; Grant-making.

Introduction

Appropriate methods to assess development effectiveness are constantly evolving, with an increasing emphasis on learning and adaptation following decades of results-based management (RBM) with its emphasis on accountability. Typically, donors provide funding to not-for-profit organisations in the state and non-state sectors to implement initiatives contributing to improved outcomes for a sustainable future. This article explores how a UK grant-making organisation, Comic Relief, which predominantly receives funding from government, the corporate sector, philanthropy and the public, is implementing a new learning agenda through the ‘I Define Me!’ (IDM) initiative. The article’s focus is on illustrating how the grant-maker supports learning through its grant-making practices and learning facilitation. The grantees are mostly in the not-for-profit sector, including international non-governmental organisation (INGO) and national non-governmental organisation (NGO).

The findings in this article are based on action research case study of one learning cohort of grantees that was established by Comic Relief. The donor, the learning facilitators and the grantees were involved in a process of action reflection on the learning journey.

The article initially presents the results-based paradigm that has been the predominant management approach in international development since the 1990s. It shows that RBM attempted to promote accountability and learning as core principles, still for various reasons, the accountability agenda ascended which stifled learning and innovation. Two possible reasons are offered. Firstly, because of its fundamental theoretical underpinning in the principal–agent theory, RBM was unable to facilitate the necessary trust to encourage learning on the ground. Further, this theory was intrinsically applied because it fitted with the existing unequal power dynamics between funders, often from the North, and grantees, often in the South. Secondly, RBM could not achieve its desired level of learning because of its mechanistic approach, based on the belief that sustainable development outputs and outcomes can be controlled through proper planning, for which organisations should be accountable. This mechanistic, bureaucratic process defined monitoring and evaluation (M&E) and the relationship between funders and donors to the detriment of trust and learning. The case study of Comic Relief’s IDM learning facilitation process, then examines the concepts of power, accountability and trust in context.

The problem which this article addresses is that whilst the RBM has the intention to encourage both learning and accountability, various factors contribute towards the dominance of upward accountability towards donors. This traps donor recipients into a bureaucratic paradigm of proving their contractual obligations, rather than allowing dynamic adaptation in the field. The effect is to stifle innovation and effectiveness, and works against meaningful and sustainable impacts. This article asks how the donor–grantee power imbalance can be narrowed to improve learning. The case of how Comic Relief supported a learning cohort with one of its grants is discussed as a possible methodology for strengthening learning. It can be argued that the Comic Relief learning agenda is influenced by the shifting paradigm away from RBM towards adaptive management (and results).

Literature review

Results-based management as a dominant paradigm

In the 1990s, RBM became the dominant paradigm within which development projects were designed, implemented and evaluated. Results-based management grew in line with the spread of New Public Management (NPM) that involved diffusing and adopting ideas from business to the public sector (Shutt 2016). This strengthened donor demands for increased transparency, effectiveness and efficiency in development aid in response to poor management and corruption, amongst other things (Kalimullah 2012). Results-based management’s intended purpose was to shift the focus on inputs and outputs to outcomes and impacts (Rist & Zall Kusek 2004 in Shutt 2016).

The evaluation community embraced this because it promised a greater focus on formative evaluation and learning (Mayne & Rist 2006; Nielsen & Ejler 2008 in Shutt 2016; Rist & Stame 2006). Donor guidelines followed. Examples are the World Banks’s influential book on 10 steps to RBM which was released in 2004 (Rist & Zall Kusek 2004), and in 2011, the United Nations Development Group produced an RBM handbook. Many other initiatives followed which reinforced RBM including the initiatives reinforcing the RBM paradigm including the 2000 Millennium Development Goals and 2005 Paris Declaration of Aid Effectiveness (Bester 2016; Shutt 2016).

Results-based management remains the dominant mode for many international organiastions, including the United States Agency for International Development (USAID), Department for International Development (DFID), the United Nations and the Organisation for Economic Cooperation and Development (OECD) (Bester 2016; Honig 2020; Shutt 2016). ‘Most United Nations entities use the tools and principles identified in the United Nations Development Programme (UNDP) RBM Handbook, adapting these to suit their mandates as required’ (Bester 2016).

This paradigm extends to INGOs and NGOs as many receive funding from the public sector, and INGOs and NGOs play an important role in international development and government services. The prominence of INGOs emerged in the early 2000s when they became a key implementing partner for many donors and funders (Anheier, Glasius & Kaldor 2001; Collingwood 2006 in Williams 2010). In many countries, including South Africa, NGOs extend government services to the most marginalised and vulnerable populations delivering critical services on government’s behalf (Southern Hemisphere 2018).

According to the UNDP (2011, cited in Bester 2016) handbook, RBM is:

[A] management strategy by which all actors, contributing directly or indirectly to achieving a set of results, ensure that their processes, products and services contribute to the achievement of desired results (outputs, outcomes and higher-level goals or impact). The actors in turn use the information and evidence on actual results to inform decision-making on the design, resourcing and delivery of programmes and activities as well as for accountability and reporting. (p. 3)

This definition highlights both RBM purposes – proving and improving.

Did results-based management encourage learning in international development?

In 2006, the OECD developed Managing for Development Results (MfDR) (Shutt 2016). The five MfDR principles identified include managing for results, learning and decision-making.

Despite learning being a key RBM principle, it is widely argued that a tension between accountability and learning exists and that accountability has won (Abrahams 2008; Honig 2020; Shutt 2016).

It can be argued that this tension is not inherent in RBM, but that it depends on purpose. A recent review found that the purpose of the use of RBM was not clear in most organisations. For some, RBM serves direction and accountability needs, and for others, the main function is learning (Vähämäki & Verger 2019). Others, however, argue that RBM’s theoretical foundation, the principal–agent paradigm, defines its implementation.

Results-based management and principal–agent theory

Results-based management has been criticised for being top-down, reflecting donor power over funding recipients (mainly developing world governments and INGOs and local NGOs). Central ideas of NPM and RBM are informed by principal–agent theory (Shutt 2016; cf. Eyben 2015):

This theory assumes that because individual agents or staff are rational actors motivated by self-interest, principals or policymakers need to design organisational structures and performance rules to create incentives that will align staff interests with the policy agenda. (p. 23)

Results-based management involves the development of systems and tools for performance measurement that regulate an agents’ behaviour in the interests of the principal (Berghmans 2016; Eyben 2015:24; Shutt 2016:23; Williams 2010:31).

As a result, the donor–recipient relationship has been defined by this accountability and control paradigm with inbuilt power dynamics (Abrahams 2008). International non-governmental organisations, NGOs and even southern governments depend on funding and are in the business of proving results. Whilst this manifests in the charity landscape in the United Kingdom, mentioned because the case described below has UK¹ grantees, it is amplified by relationships amongst mainly northern donors and southern grantees. The expectation that grantees report against pre-determined objectives and indicators (accountability paradigm), combined with unequal power relations between grant-makers and grantees, creates a culture that stifles innovation and learning (Abrahams 2008).

Results-based management as a mechanistic process

Results-based management is widely critiqued as mechanistic, assuming that change is predictable, linear and controllable. Yet, others, such as Bester (2016), argue that learning, ownership and adaptation are key features of RBM theories (Bester 2016).

Results-based management has been predominantly implemented through logical frameworks with clearly defined project objectives and pre-determined indicators against which organisations had to account, which effectively discourages learning. Abrahams (2008) argues that log-frames could act as an ‘…“agreed-upon” scheme of accountability which in its dominant mode, is mechanistic and has productivity as its ethic and power at its core’. Whilst RBM did not exclude learning, in reality, it became a bureaucratic exercise where implementing organisations proved that they had achieved their goals, rather than as a guide for reflection. The log-frame formed part of contractual agreements between donors and implementers, remaining static documents against which grantees had to report. In Guijt’s (2010, drawing on Reeler 2007) words:

[T]he ‘map’ becomes the ‘world’. And in so doing, it often turns into a rigidly followed contract that requires proof of deliverables as the heart of development effectiveness. This perspective is motivated by a need for ‘accountability’ and is driven by a logic that views development as ‘projectable change’. (p. 279)

A further problem with unequal power relations and top-down-control reporting and measurement frameworks is that accountability tends to be upward towards those with money and the power to decide its allocation, rather than to staff, partners or beneficiaries (Berghmans, Simons & Vandenabeele 2016). But what is accountability, and can one truly be accountable without learning?

Accountability

Accountability is a key international aid term. It is a strategic term rather than a neutral concept, deployed in specific social and political contexts; its use therefore serves specific needs and interests (Abrahams 2008). Results-based management became synonymous with upward accountability, deemed critical to improve trust and spending effectiveness. Keating and Thrandardottir (2017) highlight that in the 1990s, several publications questioned NGOs honourableness and effectiveness (Edwards & Hulme 1996a, 1996b; Gibelman & Gelman 2001; Smillie 1997; Sogge 1996 in Keating & Thrandardottir 2017). This resulted in a renewed focus on NGO trustworthiness, legitimacy, accountability and representativeness (Felman 2002; Holmén & Jiström 2009; Hudson 2002; Montanaro 2012 in Keating & Thrandardottir 2017).

In the United Kingdom, this focus emerged in 2010 with the emphasis on results and value for money (Shutt 2015 in Shutt 2016), and this is when the effects of RBM began to be felt by practitioners. Indicators and targets became increasingly important, even though logical frameworks had long been part of the DFID proposal negotiation processes (Holzapfel 2016 in Shutt 2016). The result has been the ascendancy of accountability over learning in the mainstream of development practice.

Trust is essential for learning, and accountability is supposed to increase trust. ‘Accountability – and the belief that communication in the form of simple measurable facts will lead to increased trust and legitimacy – drives to a large extent the results-based management agenda’ (Vähämäki & Verger 2019). However, scholars of trust argue that the opposite is, in fact, true, as discussed below.

The importance of trust as an accountability and learning incentive

If trust is critical, how is it best achieved? The proponents of the accountability agenda work implicitly with a rational model of trust and believe that donors will trust NGOs more if they have better knowledge than them – hence, more data and more reports will increase trust as will increased interaction (Keating & Thrandardottir 2017). Keating and Thrandardottir (2017) assert, however, that the causal link between increased accountability and increased trustworthiness has been insufficiently considered, arguing that the rational school of trust premise is, in fact, mistrust. ‘For social trust theorists, it is actually an open sign of distrust in NGOs’ (Keating & Thrandardottir 2017:9). In their view, the accountability agenda sustains greater distrust by promoting the idea of unfamiliarity, which makes costly and burdensome reporting requirements permanent. Further, they argue that the increase in codes and regulations, particularly in the United Kingdom, to increase NGO accountability and restore public trust in NGOs is actually having the opposite effect (see, e.g. Sir Stuart Etherington et al. 2015a in Keating & Thrandardottir 2017).

This RBM and rationality accountability paradigm with mistrust as its basis is highly problematic for promoting learning. Is there another way to view accountability? Guijt (2010) argues that accountability can be redefined to be about taking responsibility, for an organisation to be accountable to its own mission, and to do this, organisations need to learn. Guijt (2010) further emphasises that effectiveness requires both accountability and learning, and one cannot be truly accountable without learning because without learning and improvement, projects cannot be effective. This is increasingly recognised in the international development sector.

So, how can the donor–grantee power imbalance be narrowed to improve learning?

New trends in international development

The discourse is shifting to recognise that development is complex and that the results-based paradigm is unfit for purpose. Bester (2016) argues that more recent approaches to design, implementation and evaluation of development programmes, such as theory of change (ToC) and programme-driven iterative adaptation (PDIA) (Andrews & Woolcock 2012), still focus on results although they introduce innovations to improve on RBM implementation. However, proponents of these theories and tools would argue that they are based on an alternative assumption that development and change are political, unpredictable and non-linear and they rely on local leadership and relationship quality. Further, results need not be pre-determined in long-term planning and should be adjusted along the way (Shutt 2016).

Theory of change warrants further explanation as it is one of the main methodologies used in the case study to inform the learning agenda. There is no single definition of ToC, but the definition put forward in a review of ToC by Comic Relief emphasises learning (James 2011, cited in Vogel 2012):

Theory of change is an on-going process of reflection to explore change and how it happens – and what that means for the part we play in a particular context, sector and/or group of people. (p. 9)

Theory of change invites exploration and testing of the theory through learning and adaption. It is hence a good tool to use to establish a learning agenda about programme design (Britton & Thakali n.d.).

The 2014 doing development differently (DDD) conference released a manifesto, recognising that ‘many development interventions fail to address this complexity, promoting irrelevant interventions that will have little impact’ (Doing Development DIfferently Manifesto 2014). The manifesto has six principles to guide organisations DDD, the fourth of which is that organisations should ‘blend design and implementation through rapid planning, action, reflection and revision cycles (drawing on local knowledge, feedback and energy) to foster learning from success and failure’.

A recent addition to this school of thought is Navigation by Judgment (Honig 2020).

These alternative approaches reverse the principal–agent dynamic, giving the principal more power to respond to circumstances on the ground. They emphasise adapting to local context and capacities, and practitioner experience as an important source of evidence, alongside data from M&E. Hence, they require trust in the implementing organisations to make the right judgements based on their interpretation of collected data and the situation they find themselves in.

Principle 4 of the DDD manifesto is the most relevant for this article as it speaks directly to learning. But how do we understand learning? This is discussed in the following section.

Learning

This section provides insight into the factors that the learning facilitation team considered when designing the learning facilitation for the ‘IDM’ programme. Three important factors for learning are raised, namely the use of ToC as a tool for critical reflection, the importance of dealing with power dynamics to foster a climate conducive to learning and the value of practitioner experience as one source of evidence. Learning is about assessing project results to inform programme improvement or increase knowledge in a subject area. Traditionally, monitoring has been about tracking projects that rigidly adhere to pre-determined plans set out in RBM frameworks, like log-frames. Learning is usually focused on programme theory, relying on tools such as ToC to address social change complexity (Guijt 2010).

USAID’s Learning Lab (USAID n.d.) says learning is about asking the most important questions and finding answers relevant to decision-making.² Guijt adds the dimension of learning being about critical reflection, not just pragmatic decision-making. In this sense (Guijt 2010):

[L]earning requires capacities for critical reflection, identifying assumptions, seeking evidence about what is going well or not, analysing multiple lines of evidence, relating evidence to expectations and analysing and negotiating possible consequences. These processes all require connecting people and their perspectives. Therefore, the capacity to deal with power dynamics becomes essential. (p. 282)

This type of learning requires more direct practitioner involvement in programme change than is traditionally acceptable in top-down management (Guijt 2015; Gonsalvez De Asis & Woolcock 2015 in Shutt 2016).

Practitioner experience has often not been considered a legitimate source of knowledge in the RBM paradigm. Views on what constitutes quality evidence are central to conversations about the results agenda’s strengths and weaknesses, and is influenced by epistemological beliefs about the nature of knowledge.

There is increasing interest from funders who are interested in supporting learning. For example, in 2015, The William and Flora Hewlett Foundation commissioned a study to inform grant makers how to support more effective learning (Ross 2015). In this article, we present the case of a learning initiative by Comic Relief, a UK-based fundraiser and grant-maker.

Methodology

Recently, Comic Relief has experimented with developing a collective element to some of its work with grantees to encourage a learning focus, based on the hypothesis that it is necessary to create conditions for projects to have the impetus for ongoing learning and adaptation. To do so, Comic Relief realised that it had to create spaces outside of projects where hard-pressed project workers and managers can step back and refocus on the bigger picture. It was also important to be in a different power context with the funder than in individual project accountability, and be stimulated by others, which also facilitates cross-fertilisation of practices amongst projects.

Comic Relief has tried various configurations to set up and manage this collective element. Models included being led by one grantee, collective responsibility amongst grantees and being led by a Comic Relief-commissioned consultancy and a grantee-commissioned consultancy. The trials had varying results related to power dynamics amongst projects and organisations, capacity and ownership, content focus and contexts. The most successful was the Comic Relief-commissioned learning coordinator (led by Southern Hemisphere in partnership with Framework) for the IDM initiative. Comic Relief awarded nine grants to multi-partner initiatives in three countries (South Africa, the United Kingdom and Colombia), which all worked with gang-affected young women and girls over 3 years (2017–2020). The organisations in the cohort were diverse and included three local NGOs or charities, one university, four INGOs and one local government group with nine civil society partners.

Each project was asked to select two to three participants for participation in the process drawn from each of the partner organisations. Typically, each project had at least one person who was consistent throughout, and invited other partners to the various events. Most of the learning cohorts were managers and frontline workers, except for the two academics.

The case study presented below describes the learning facilitation role implemented over these 3 years. The lessons identified are based on the experience and reflection of the learning team and the grantees throughout the process. The learning facilitation team comprised two people from Comic Relief (the grant-manager and the evaluation and learning lead³) and the learning coordination team, a joint venture of two organisations, namely Southern Hemisphere and Framework. The team also included a leading academic in the field of criminology as our subject expert.⁴ The learning facilitation team and the grantees periodically reflected on the learning journey based on an action research methodology. Action research is an evaluation approach that is described as a cycle of planning, acting, observing and reflecting (Life Changes Trust Evaluation Toolbook).

The learning facilitation process is described in detail in the section below, as the process itself is a finding of this case. The process itself was dynamic and emerged based on reflection. There were two main questions that the learning facilitation team were interested in learning about through this process (1) what does it take to encourage learning amongst a cohort of grantees given the power relationships and RBM paradigm that is still prevalent in the sector and (2) what did we learn about working with girls and young women who are gang-affected. This article addresses the first question. The second question is dealt with in a sector brief that targets decision makers in the sector working with people who are gang-affected.

The learning facilitation process generated lessons for both questions above through two main methodologies: firstly, the learning facilitation team synthesised data from the monitoring reports of the grantees. Projects reported bi-annually on their monitoring data and were asked to reflect on their learning questions in the reports. The synthesised information was presented to and discussed with the learning cohort; therefore, grantees were aware of trends and themes emerging across projects. The second method was through interactive learning workshops (virtual and face to face). There was a high degree of participant involvement in these sessions, which used facilitation methods such as Open Space Technology, in which participants shape the agenda and engage in open discussion and reflection. The lessons from each session were captured in reports using the learning framework and were shared with the participants. All the process and the reports were provided in Spanish and English.

These sessions were guided by the learning framework that was established in a participatory way by the participants in the first meeting of all the grantees in London in 2017. Theory of change methodology was used to guide this process. The ToC process implemented was a combination of the processes suggested in the Humanistisch Instituut voor Ontwikkelingssamenwerking (Humanist Institute for Cooperation with Developing Countries [HIVOS]) ToC Guideline (Van Es, Guijt & Vogel 2012), ActKnowledge Facilitators Source Book (Taplin & Rasic 2012), the Community Builders Guide to ToC (Anderson n.d.) and the Southern Hemisphere Training Manual which draws on all of the above (2014).

No ethical review was conducted for this project as it was not intended to be a research project at the outset, but all the projects gave consent to use their information to share the lessons. The names of the projects and the grantees are withheld.

Findings and discussion

The case study: Comic relief’s experimentation with learning

Comic Relief raises money from the British public and through partnerships with other funders and corporates. Comic Relief then makes a broad range of grants to charities across the United Kingdom, sub-Saharan Africa and South Asia to support work in gender justice, early childhood development, informal settlements and homelessness and mental health. Comic Relief has always focused on encouraging and facilitating grantee learning, largely by investing in individual grantee monitoring, evaluation and learning (MEL) capacity and systems. However, despite allowing flexibility to change approaches, budgets, outcomes and indicators based on project experience, in many cases, M&E still reverts to trying to ‘prove’ a vision of success. Grantees chase targets set in an implementation vacuum at the project start, which become increasingly irrelevant to their work’s context and realities. Analysis of relevant data, reflecting on experience, acknowledging challenges and learning and adapting projects to be as effective as possible (which Guijt [2010] argues should be the basis of accountability) is therefore lost.

The learning journey of the ‘I Define Me!’ project

This article reflects on the experience of implementing the IDM initiative’s learning coordination component, covering the process’s practice and outcomes. Introducing learning as a grant-making support component was new to grantees who were used to a paradigm of accountability and reporting, not reflection and learning. We encountered various challenges identified by Guijt (2010) that block learning; we share how we designed the process to overcome and address them to unlock learning potential and enable organisations to learn.

The learning facilitators accompanied the cohort through the grant period. Working with gang-affected girls and young women was a new thematic area for Comic Relief, which appointed a learning coordination team to support programme development through collective learning and to increase knowledge and understanding about working with the target groups. The learning facilitators’ main role was to support projects as a learning cohort, promote learning amongst projects and understand areas of crossover and divergence. Learning was facilitated through face-to-face events, virtual workshops and communications products. The learning facilitators also undertook wider learning work by analysing and reporting back to grantees on discussions from the learning process and project reports. Hence, the process included facilitation, synthesis research, documentation and communication.

Learning facilitation methodology

The learning facilitation process took place over 3 years. Grantees and Comic Relief gathered three times for learning processes: once at the start in London, once after 18 months in Bogota and once towards the end of the 3-year period in Cape Town. All workshops were facilitated using creative, analytical and experiential techniques and we created space for participants to co-create the agenda for the mid-term and final workshops. Learning facilitators coordinated the events.

The learning facilitators used the assumptions in the ToC to develop a learning framework (see Box 1 for the learning questions). A learning framework includes learning questions, principles and other factors that set the environment for learning. The learning framework contained questions about context, intervention design, methodological effectiveness and outcomes. These are presented in Box 1.

A generic ToC and learning framework were created with grantees⁵ at the kick-off workshop in London in 2017. Learning facilitators were assigned to each grantee organisation to support their learning journeys. Back in their home countries, a learning facilitator helped each organisation develop its own ToC and learning framework, aligned to the generic one. Learning questions were mainly derived from the assumptions in their programme theory. Following this, grantees submitted start-up forms to Comic Relief to receive their first grant. At this stage, they could reinterpret their project design based on work with learning facilitators and discussions with grant managers.

After the mid-term workshop, some participants requested a less linear model of understanding change than in our logic model. We therefore developed a rich picture, which also helped capture the complexity of the projects’ experiences. Interestingly, most projects struggled to engage fully with the ToC, except for the academic project. It was because the ToC was embedded in the learning questions that it was still a central part of the learning process.

A clear boundary was drawn between learning facilitator and project M&E activities. Whilst learning facilitators helped projects to reflect on whether they were collecting useful data for learning, projects had to design and implement monitoring systems. The Comic Relief grant funded M&E work.

As this is a new area of work, with little evidence or social theory about what works, it required a high degree of experimentation and innovation. Implementation reflected the complexity of development as discussed in the literature review. Target groups were unpredictable and, as projects discovered, they tend to live ‘chaotic and complicated lives’, reflective of the contexts projects were working in. In Cape Town, for example, gang violence can disrupt activities at short notice and for lengthy periods, meaning projects need to be able to decide and use their best judgement to be effective. Honig found ‘…that agencies that have greater likelihood of navigating by judgment see much less difference in their performance as environments become more unpredictable than do agencies which use more top-down control’ (Honig 2020:3).

Comic Relief’s support for learning encourages projects to use judgement, based on practitioner experience and monitoring data; therefore, frontline workers – those actually implementing projects – were involved in a process of action and reflection. Honig (2020) describes the value as follows:

[I]f we Navigate by Judgment, if we empower those closest to the ground, we’re going to get more agent initiative. We’re going to get more use of soft information – things that agents can see and learn, but can’t verify with numbers. We’re going to get more flexibility (p. 1).

The challenges for learning and how we responded

Guijt (2010) recognises that even if funders want adaptive projects and to improve implementation from learning, there are several constraints. Common constraints raised by organisations include time, capacity, M&E trends, context constraints and organisational culture. Below the key constraints and how the learning facilitation design addressed them are highlighted.

Rigid planning processes based on idea of predictable change

This initiative was launched based on little practice or knowledge of working with gang-affected girls and women, underpinned by an openness to experiment, test and adapt over time. The ToCs were not set in stone, with scope to change activities as projects became more familiar with beneficiaries and partners. We also identified ToC assumptions as part, which became the basis for identifying learning questions to guide the monitoring and learning process.

At reporting points, Comic Relief allows grantees to change activities, outcomes and indicators based on learning and adaptation, but few go beyond superficial shifts. Crucially, this initiative provided explicit, individual and collective opportunities and prompts to reflect on experiences and learning questions through individual conversations with learning facilitators before annual reporting and collective learning events every 18 months. This led to revising the original collective ToC to capture the non-linear and diverse experiences of the girls and young women. This adaptability and reflection at the collective level were replicated at project level with more detailed and ongoing shifts in project approaches and plans (and indicators) to recognise the fluidity, messiness and patience involved. For the first year or two years, this flexibility and change were based not only on formal outcome data but also on implementation experience and principles underpinning the work (building trust with beneficiaries, the time this takes, ensuring methods of data collection did not damage this, etc.). We discovered projects that are often pushed to produce outcome results too fast, rather than recognising that outcomes take time to emerge, whilst questions of project design, implementation practice and learning are often ignored or undervalued when discussing M&E.

Projects commented that this felt like a different approach to their other work – whilst they have a clear picture of what they want to achieve (overall outcomes of the collective and individual ToCs), it was open on how they get there, and therefore, it developed as a girl-centred rather than a paper-centred initiative.

Practical limitations (time to learn)

‘We don’t have time!’ (Guijt 2010:279) is always a significant challenge at all levels. We approached this at cohort level by demonstrating the investment in learning coordination. We were not hugely intensive with each project given the international nature and budget constraints, but we provided specific space at key points outside the day-to-day of projects, facilitated by the learning facilitators.

The process created time to think – workshops, webinars and individual conversations with and between the grantees. The projects valued the time to stand back and reflect outside of daily work. It was important that learning sessions occurred at critical points in the project cycle with enough to reflect meaningfully at a collective level, and that they mirrored the pace of project implementation and change rather than forcing quick learning to emerge. This required patience from Comic Relief – in the first 18 months, we were concerned about having little engagement or few specific things to show from the cohort, but waiting 18 months to bring people together meant they had much to share and consider so that the mid-term event was a great success. This triggered many of the projects to recognise the value of making time to reflect with colleagues. Since then, several projects have established regular team reflection processes, including frontline practitioners and senior managers to bring in different perspectives. In some cases, this has influenced how organisations approach other unrelated projects.

Capacity constraints (skills in ‘transformational discussions’ vs statistical analysis of monitoring data)

Most of the participants in our cohort, including the MEL staff, were not practiced in learning and reflection, and the process was designed to help them develop this way of thinking. As Guijt highlights, ‘Clearly different skills are needed to do statistical analysis than to facilitate transformational conversations in organisations…’ (Guijt 2010:279).

The learning coordination role deliberately did not support individual projects’ M&E systems or data collection and analysis but facilitated and elicited learning discussions, valuing whatever information or data informs that, including project workers’ observations, experiences, challenges and formal data projects gather. We also allowed grantees to direct discussions, many of which concerned ways of working and the principles underpinning this work. As noted above, demonstrating the usefulness of such discussions at a cohort level has led to more reflective discussions and practice at individual project level. Through the learning facilitation process, we helped participants learn how to learn – how to engage in reflective practice and use evaluative thinking.

Whilst the process has been valuable (and arguably has led to more embedded use of learning), the degree to which organisations produce clear evidence-based outcomes through more traditionally accepted data is variable. Towards the project end, we had to remind projects to focus on capturing actual evidence of change, as opposed to measuring quantitative targets. It became evident that the hard boundary between the learning facilitation and the M&E system support was potentially problematic, as the learning facilitators could not ensure that projects were collecting evidence to support their experiential learning.

Economic and political trends (being comfortable in outcomes vs impact space)

Organisations compete for scarce resources and need to produce hard data to show success, rather than messy stories of change (Guijt 2010).

As the project focus was largely new for most of the organisations (many had worked with young men around gangs but not young women), it was not about proving existing work approaches and large beneficiary numbers. It was about reflecting on how to adapt existing models, engaging with other projects using different approaches and learning from one another over time. Comic Relief did not push for large beneficiary numbers, emphasising depth over breadth. In fact, some projects reduced numbers as the intensity and continuity of work required with individuals became clear. It became evident, for example, that the work needed to be trauma-informed; the time it took to build trust with young women who had experienced high degrees of trauma meant that they had to lengthen the time they worked with groups, reducing their intake. Revising numbers downwards became desirable and optimal over scale. This flexibility on the part of the donor was an essential part of the learning process and contributed to the building of trust that was identified as being so critical in the literature review. This is elaborated further below.

Context constraints (and incentives) – (incentivising learning)

As noted above, Comic Relief has always enabled flexibility in grantee activity plans, budgets, outcomes and indicators but has not always succeeded in creating a relationship that makes grantees feel comfortable doing this (also recognising that many NGOs or charities have the rigid, linear accountability approach integrated into their internal processes in response to funding requirements). The challenge is that ‘Rigid accountability systems hinder learning’ (Guijt 2010:280).

The learning coordination role enabled projects to trust that Comic Relief was genuine about learning and that they really could change their activities, outcomes and corresponding indicators.

Engaging in honest and reflective spaces with the grant-maker present emboldened organisations to open up and engage evaluatively with their work. The space to talk about learning and the provision of feedback and reflection incentivised projects to focus on learning over performance. Comic Relief, however, did not make the projects accountable for learning. The learning process was incentivised through the learning events and the luxury of having time and space to think, share and have fun in the process. We often forget to ensure that people delivering work on the ground are taken care of and have opportunities for growth. This process enabled both.

Learning may be considered a luxury most funders and projects cannot afford (in time or dollars). But, although we provided spaces to learn, we demanded little organisational time and made few demands on grantees. In terms of money, the cost was small relative to the overall budget, but the benefit for Comic Relief was tremendous.

The learning coordination’s outcome is that Comic Relief has seen far more change in project design and implementation in this cohort than it traditionally sees in grantees, for example, projects expanding target groups (e.g. recognising the need to work with young women’s families to effect change) or introducing new methods such as wilderness therapy or creative arts. Everything has changed far more than in the average grant Comic Relief has seen in terms of practice, approach, indicators and budgets. This confirms that providing the space for learning encourages adaption and innovation, which, in turn, are expected to increase effectiveness.

Organisational culture (resources and responsibilities allocated to accountability and learning – both embedded in specifications and project design)

Comic Relief encourages organisations to invest in MEL capacity, which it happily funds. Grantees can fund M&E posts and Comic Relief funds the whole M&E cycle so that projects have sufficient resources to collect, capture, store, analyse and report on relevant data. Grantees were funded to participate in the learning process and Comic Relief contracted and paid for the learning coordinators. Hence, the grant included funds for both accountability and learning, which is a practice supported by Guijt – ‘Resources and responsibilities need to be allocated to accountability and learning’ (Guijt 2010:280).

As a funder, Comic Relief cannot force organisational learning. For most grantees, organisational learning was not part of their organisational culture as they implement within a paradigm of upward accountability. However, through this learning process, they experienced the power and value of learning; many have taken these practices back to their organisations. However, organisational learning needs leadership from the top and clarity on how learning is to be used. This was easier for relatively new projects with more flexibility to learn.

Addressing power relations

It is critical to address RBM power dynamics to break the status quo that locks the system into forms of accountability that inhibit learning (Guijt 2010). We were conscious throughout the process of the grant-maker–grantee power dynamics. These power dynamics are the focus of discussions in many donors, Not-for-profit organisation (NPO) forums, and were raised by Abrahams (2008) as a critical concern especially when individuals do not question the power that they wield through the intellectual, cultural, material and financial resources that they control.

Having an independent facilitation team meant Comic Relief did not facilitate the sessions that helped normalise the power dynamics – it could fade into the background and leave the room if necessary. The grant-maker asked grantees’ permission to attend the learning events, clarifying that it would leave the room if asked to do so. It was, however, important that it was engaged in the conversations and willing to learn with the participants as it helped to establish an honest and trusting relationship that created the right conditions for learning. It was also important that the learning facilitator role was independent of the grant-maker, and could have prompting conversations with projects from a pure learning perspective.

A ‘contract’ for learning was established during the first learning event, which was included in the learning framework. This helped establish group norms around learning, creating the basis for trust and equity amongst the group.

Conclusion

Encouraging learning is essential for effective and sustainable development; more grant-makers could actively support learning initiatives and organisations could establish internal learning processes. A key lesson from this process is that the learning focus can be transformative for individuals and organisations. However, a grant-maker consciously needs to encourage learning conditions. Organisations must ‘unlearn’ the need to prove success, which has been compounded by years of operating in the RBM paradigm that is described in the literature review. Rather, they need to learn to honestly assess their strengths and challenges. By trusting organisations to learn and encourage them to ‘Navigate by Judgement’, Comic Relief shifted the principal–agent paradigm, showing that projects they could adapt and improve. Organisations need to ‘learn how to learn’, and ‘unlearn’ to just report. The learning facilitators’ role is critical; it provides structure, created through the ToC and learning framework co-created with the participants, and a safe space to enable the learning process. By focusing on learning questions that addressed assumptions, participants learned not to focus on questions like ‘Did we do this?’ but rather on, ‘did this assumption hold true in our experience or did something else happen?’

Safe spaces were created through the facilitation methods and the relationships established. Coming together for learning events was important, as was having learning facilitators supporting projects individually. Synthesising the grantees’ annual reports and feeding back to them was also important to show we took their reporting seriously. It enabled us to identify common themes and trends and helped them reflect on their practice and find discussion points, which encouraged them to write better and more honest reports. The synthesis work, learning event reports and production of communications materials by the learning facilitators enabled the learning from this group of projects to be shared more broadly with a wider sector audience.

The learning facilitation role played an important part in enabling the projects to implement adaption and innovation.

Lastly, it is critical that funders demonstrate willingness to learn by being involved in discussions about learning. Funders need to show participants what they are interested in learning about, whilst at the same time allowing themselves to be led by the participants’ interests.

Acknowledgements

The authors acknowledge the other learning facilitators, Orla Cronin, Elena Mancebo, Wilma Wessels-Ziervogel and Bill Sterland, for their co-creation of the learning facilitation process. They thank Anya Stern, Grant Manager at Comic Relief for her partnership in this journey. They also thank all the participants who provided their insights and reflection on the learning process.

The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this research article.

D.L. and J.G.-S. contributed equally to this research article.

This article followed all ethical standards for research without direct contact with human or animal subjects.

The research received no specific grant from any funding agency in the public, commercial or not-for-profit sector. The learning coordination task itself was funded by Comic Relief.

Data sharing is not applicable to this article as no new data were created or analysed in this study.

The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of any affiliated agency of the authors.

References

Footnotes

1. The Sir Stuart Etherington et al. report entitled ‘Regulating Fundraising for the Future: Trust in Charities, Confidence in Fundraising Regulations’ is a response to a legitimacy crisis of UK Charitable organisations, NGOs and INGOs.

2. See https://usaidlearninglab.org/qrg/understanding-cla-0.

3. The evaluation and learning lead from Comic Relief is Jake Grout-Smith the co-author of this chapter.

4. The author of this article, Dena Lomofsky, headed the learning facilitation team.

5. The 3-day workshop was attended by two to three people from each of the nine grantee organisations, including frontline workers and M&E and fundraising staff.