Abstract
Background: A complexity-aware approach, realist evaluation is ideal for norms-shifting interventions (NSIs), which are not well-understood but gaining prominence in behaviour change programming in Africa and globally to foster enabling socionormative environments that sustain behaviour change. A new application of realist evaluation to NSIs uses an adapted approach employing realism values that is suitable for social and behaviour change (SBC) programme evaluation more generally.
Objectives: This article shares the authors’ reflections on tailoring realist evaluation approaches for use with community-based norms-shifting programmes. It describes how realist evaluation enables co-building of programme theory that conceptually underpins NSIs, guides evaluation efforts and yields benefits beyond theory-proving.
Method: Two NSIs in Niger and Senegal illustrate how locally refined theories of change (TOC) and identification of evidence gaps in causal pathways guided a series of rapid programme and quasi-experimental outcome studies. Over two years externally and internally led studies assessed intermediate or mediating norms-shifting effects and outcomes comprising the realist evaluation. Studies drew from experiential, existing and new data.
Results: The tailored approach created a co-owned evaluation, from joint exploration of SBC theory to review of evidence generation. Five values applied to the research–practice partnerships reinforced a realist perspective: participatory, complexity, shared ownership, practice-oriented and valuing all forms of data.
Conclusion: Bounded by TOC exploration for programme inquiry, realist evaluation embeds learning and assessment concretely into local programming and knowledge building. Integrating evaluation practice with realism values creates a nexus and a unique and significant dynamic between programme implementers and evaluators that transcends NSI research and programme practice.
Keywords: realist evaluation; social and behaviour change; health; norms-shifting interventions; Niger; Senegal.
Background
Realist evaluation represents a departure from earlier evaluation approaches because it more fully recognises the complexity of interventions and their receiving environment to explain outcomes. It is anchored in realism, a sociological perspective that focuses on the process or mechanics of change to explain how cause leads to effect over time, leading to a body of scientific knowledge and meaning (Pawson & Tilley 1997). Positivism is the contrasting perspective that focuses on observing the direct relationship or measurement between cause and effect to create knowledge and meaning, often associated with evaluation focused on pre- and postintervention outcomes. In realist evaluation, the articulation of programme theory guides evaluation to build an understanding of the mechanisms through which programmes achieve change, accounting for context, changing conditions and time (Pawson & Tilley 1997). Over time, using a cyclical, knowledge-building approach, realist evaluation refines the programme theory, intervention and expected outcomes of complex programmes as new information is learned (Danks 2017; Duncan et al. 2017; Salter & Kothari 2014; Vugts et al. 2017).
Because of the realist focus on understanding complexity, social science researchers are increasingly interested in realist evaluation for projects, programmes and complex public policy analysis. Twenty-five years after it emerged, it is becoming more widely accepted as an evaluation approach with applications in health, agriculture, social justice, environment and other sectors (Jagosh & Tilley 2016; Tilley 2017). It has demonstrated its ability and utility to provide information to adapt and improve programming, as it helps explain the possible reasons why interventions work, or are not working, and their value (Liu 2017). It can be applied to many types of evaluation, from formative assessments to assessing multilevel interventions, from interventions not yet well tested to those in full implementation (Dossou & Marchal 2017; Vugts et al. 2017). It is increasingly used on the African continent (Mbava & Chapman 2020), although its application in Africa has been hampered by the relative lack of publications developing programme theories of change (TOC), analytical frameworks, evaluation guidance and study reports that provide examples of realist evaluation in practice (Robert & Ridde 2013).
Realist evaluation insists on more profound levels of exploration of change pathways and learning. The resulting evidence has been helpful in programmes working with complex social phenomena such as sexual violence (Marchal 2017; Rayment-McHugh 2017), unintended teen pregnancies (Aslam et al. 2017) and community-based maternal health programmes (Mathias et al. 2018). It can systematise learning within programmes. Programme implementers and decision-makers refine programme theory (Danks 2017) and become more critical users of data for programme decision-making as new evidence emerges from the reanalysis of data or new study data. The approach creates an avenue for programme implementers, stakeholders and evaluators to inform and reflect on the intervention and better understand how an intervention leads to change.
The authors’ 2021 scoping review on realist evaluation applications to preventive health programmes in lower- and middle-income countries indicated that realist evaluation is not widely applied to social and behaviour change (SBC) and behavioural science programmes, even when such evaluation would be very useful for SBC interventions, which are complex interventions implemented in complex social contexts. Relatively few SBC programmes are built on TOC (Davis et al. 2015), a necessary starting point for realist evaluation. Also, while SBC programme practice increasingly embraces complexity thinking and responsive adaptation approaches, complexity-aware evaluation is rarer (CORE Group 2021).
Norms-shifting interventions (NSI) are SBC interventions that identify social norms and promote collective change by encouraging communities to reflect on and question norms related to unhealthy behaviours. Social norms, or the informal rules that influence people’s behaviour, have an essential role in shaping the behaviour of people, and NSI strategies complement other SBC programme strategies, such as those focused on changing attitudes or providing structural support to encourage behaviour change (The Learning Collaborative to Advance Normative Change 2017). Norms-shifting interventions are gaining traction in development circles, with concurrent inroads in relation to how to evaluate the norms-shifting outcome. The authors’ experience is that NSI TOCs focus more on individual change. They therefore miss the intermediate effects that link programme activities to normative change outcomes and that clarify how norms shifting complements individual change outcomes (Nguyen et al. 2019). Programmes may appear to be working as expected on an individual level. Still, it may be unclear if the programme reaches key social influencers or the broader community needed to achieve community norms change.
Together, this represents a missed opportunity to understand if programmes are achieving change at the individual and social levels. Realist evaluation can help one understand in a programme-practical way how NSIs work. That is, it can generate understanding that is relevant and usable by programme implementers, and it is not only theoretical by unpacking how NSIs theorise and lead to behaviour change.
In 2015, as the United States Agency for International Development (USAID)-funded Passages Project (2015–2022) was beginning, multiple theories existed about how norms, among other individual and structural determinants, led to behaviour change (The Learning Collaborative to Advance Normative Change 2019). Yet there was limited understanding of how norms shifting could be influenced by programmes, including which SBC strategies effectively foster norms shifting, how norms shifting is related to other drivers of behaviours and outcomes and whether norms shifting leads to behaviour change. The Passages Project chose realist evaluation to systematically build understanding and evidence of how NSIs work in complex contexts by examining the relationship of context, mechanisms and outcomes. The project included six implementation and scale-up interventions, two of which are used as case examples in this article.
The findings of the realist evaluations are beyond the scope of this reflection article, yet it can be said that the results across the six NSI evaluations confirmed with some variations what theory would expect: quasi-experimental outcome evaluation studies showed the NSIs achieved individual and normative changes predicted in their theories, with some variations. Programme studies to learn about the intermediate or mediating effects of project activities confirmed expected changes along change pathways. Table 2 shares some high-level findings. Interested readers will find outcome evaluation study reports on the Passages website alongside many of the programme studies.
This article shares the authors’ learning about applying a realist evaluation approach. It describes how they tailored realist evaluation for use with community NSI programmes having a limited theory-to-practice evidence base. It explores how realist evaluation enabled co-building of programme theory that conceptually underpins NSIs, reflects on why it worked well and how the evaluation process yielded benefits beyond evidence-based theory-proving.
Method
Interventions being evaluated
Across interventions, Passages applied a systematic co-learning approach over a two-year evaluation period starting with building partnerships, developing a programme TOC that included norms-shifting pathways, answering key gaps in pathway evidence and evaluating intervention effects. Case examples of two norms-shifting interventions that underwent realist evaluations are described here and summarised in Table 1. Both interventions operate in rural communities to improve the normative environment supportive of behaviours leading to good reproductive health outcomes. The Husbands’ Schools intervention, which operates in several thousand communities throughout Niger, aims to increase women’s low use of reproductive health services. In many Nigerien communities, norms of unquestioned male authority, a lack of couple communication and joint decision-making and men and women’s distrust of antenatal, delivery and family planning services serve to limit the success of normative and behaviour change efforts. The Girls Holistic Development intervention, which operates in one southern region in Senegal, aims to improve adolescent girls’ health and school outcomes and reduce the practice of child marriage and related teen pregnancy. In many Senegalese communities, older women’s influential advisory and support role in child-raising has diminished compared with earlier times. The expectation that boys should attend school and prepare for breadwinner roles contrasts with the expectation that girls should help with chores and prepare for marriage. These community norms serve to check SBC efforts.
| TABLE 1: Norms-shifting interventions that underwent realist evaluations. |
As NSI programme theories would predict, both interventions’ activities are less focused on building knowledge and more on engaging people in discussion to examine prevailing beliefs in light of current issues that lead to new ideas and possibilities of ‘being’. Project extension agents and trained volunteers (social change agents) use a mix of group reflective dialogues and role-modelling to break taboos, allow debate on community expectations of appropriate behaviours for women and girls and men and boys and discussion of a vision for the future. Over time, new change agents and influencers emerge to encourage others in the community to adopt new ideas and behaviours. Eventually, enough people are exposed to and internalise new ideas and desire change that a tipping point of agreement on a new norm or acceptable behaviour is reached.
Realist evaluation scope
The application of realist evaluations under the Passages project (Figure 1) followed the prescribed realist evaluation approach, emphasising an iterative and interactive process over two years. The participatory development of programme TOC was the starting point of the evaluation and related studies. The role of Passages’ staff was two-fold – as external evaluators guiding the different evaluation studies and as co-facilitators with NSI managers of the realist evaluation process that began with theory-building. In this section, a brief description is given of how TOCs became the organising construct with evaluators and NSI staff and stakeholders in the evaluation.
 |
FIGURE 1: Realist evaluation of norms-shifting interventions: Iterative and collaborative generation, accumulation and utilisation of evidence. |
|
In the opening workshop, Passages evaluators worked alongside local stakeholders with operational knowledge of the NSIs – project implementors, monitoring and evaluation (M&E) staff, local health, education and religious stakeholders – to refine the project TOC. Building programme TOCs together created a discussion space that welcomed different perspectives and built theoretical consensus on how the NSI worked. Examining the causal pathways, the group identified evidence gaps, which led to subsequent questions to guide realist evaluation inquiry.
Over two years, Passages evaluators, consultants and NSI staff completed a series of jointly defined evaluation studies to answer different evaluation questions. The studies used a range of data types based on an established data rule: all types of data could be used for evidence, from project activity monitoring systems and existing studies to new data that would be collected in rapid and quasi-experimental evaluation studies.
Every six to eight months, realist-evaluation workshops allowed project staff, local stakeholders and Passages evaluators to review and analyse new findings in light of the TOC. In the meeting that closed the evaluations, a cumulative review with the initial stakeholder group of study questions and findings led to adjustments to TOCs. Dissemination meetings with national stakeholders created momentum to continue advancing the NSIs’ scale-up in Niger and Senegal, given the new evidence and deepened understanding of how the projects worked to foster social change that influenced behaviours.
Ethical considerations
This article followed all ethical standards for research without direct contact with human or animal subjects.
Results and discussion
Several critical adaptations were made to tailor the realist evaluation approach to community-based interventions with relatively small databases and to improve relevance to involved stakeholders. Firstly, there will be a discussion of the decisions relating to the process of initial TOC building and testing evidence gaps (hypotheses) reflected by the causal pathways. An explanation will follow regarding how the evaluation approach integrated realist values into evaluation processes that helped to equalise the co-exploration of change processes by evaluators, implementers and local stakeholders. The section closes by offering reflections on the programme-practical and sometimes unexpected effects of using realist evaluation to build an understanding of how NSIs lead to norms change.
Tailoring realist evaluation to community-based norm-change interventions
While there are many ways to visually depict TOCs, it was found to be crucial to work with the type of programme theory of change that shows change pathways from programme activities to their intermediate or mediating effects on programme outcomes. This depiction of programme theory helps to unpack the implementation box to show pathways and their interconnections leading to change. The decision was also made to group project activities into components of similar activities, instead of examining pathways of each activity separately within a TOC. This provided a way to engage programme stakeholders in thinking about how an intervention works without getting into programme or evaluation ‘weeds’. This programme-practical approach allowed realist evaluation to move from complex theory and mapping to a practical, conceptually grounded, collaborative evaluation approach.
The initial workshop focused on TOC building. The original TOCs indicated only behaviour change outcomes; the expected normative changes were not articulated or were vaguely defined. During the workshop, project staff and local stakeholders and evaluators added missing outcomes to TOCs. (See Figure 2 example from the Husbands’ Schools intervention, which added an enabling environment outcome during the TOC workshop.) Related workshop discussions on how normative shifts occurred because of project activities helped clarify how the change mechanisms, produced from activity implementation, led to community reactions and actions (intermediate or mediating effects) that facilitated norms shift. The pathways discussions led to new understandings, sometimes revelatory, by implementers, local stakeholders and evaluator-researchers.
 |
FIGURE 2: Programme theories of change developed and analysed by stakeholders of Husbands’ Schools (dashed boxes reflect evidence gaps). |
|
Both Husbands’ Schools and Girls Holistic Development interventions had previously undergone mixed methods, nonexperimental evaluations focused on individual behaviour change outcomes. These evaluations showed they were effective in behaviour change, with some anecdotal evidence of normative change. Given this earlier evidence of effectiveness, the researchers decided not to test alternative hypotheses for each component as is prescribed in some realist evaluation guidance. Instead, participants identified evidence gaps in pathways, and several studies were developed to assess whether change was occurring along those paths as expected by programme theory. This allowed prioritisation of evaluation questions. The dashed boxes in Figure 2, for example, show evidence gaps that were not supported by stakeholders’ own experience-based knowledge or past studies. They guided evaluation research (learning) questions.
Intentional incorporation of values to reinforce realism perspectives
In sociology, a realist perspective accepts that social facts are just as real as physical facts and can be known by blending theory with observation, measurement and empirical studies. Accordingly, several ideas underlie realist evaluation. Social and structural factors affect how programmes work; multiple realities and truths exist; and social systems are porous and flexible. The approach recognises that change mechanisms may not be visible, but they can be understood as the combination of resources and how they change community reasoning. Finally, context influences how mechanisms operate (Westhorp 2014).
To achieve a realist perspective, the researchers’ evaluation–practice partnership intentionally applied these values throughout their tailored approach to realist evaluation: participatory, complexity, shared ownership, practice-oriented, and valuing all forms of data. The participatory value was visible in the structure whereby everyone’s voice was valued, all stayed present during meetings and space was given for all to speak. Reflecting the complexity of social change meant being flexible in working with the expected and unexpected pathway findings and working with stakeholder knowledge of social systems in which NSIs operated. Shared ownership included sharing decision-making across organisational hierarchies and geographies and involving crucial stakeholders in all evaluation stages (design, implementation, analysis and dissemination). Learnings that advanced the project or future performance of related projects were the highest priority to be practice oriented. Finally, all forms of data were valued, existing and new, experiential and empirical, repurposed and newly generated. These five values were not named in this way at the outset but were reflected in the processes.
In practice, this meant a participatory, co-learning approach over two years to welcome into an evaluation process the NSI staff and stakeholders who have a deep understanding of how NSI activities interact with the lived realities of their respective societies. Norms-shifting intervention staff and stakeholders included people selected for their involvement in and knowledge of the programme design and implementation, interaction with community members who participate in the programme and involvement in research. In meetings, facilitation was shared by project staff and Passages evaluators, with attention and emphasis on sharing and collaboratively learning, reinforcing the realism concept that multiple realities exist and thus no one person could hold the ‘right’ answer.
Different types of data and data sources represented a mosaic of information that could be used to answer evaluation questions. The ways in which the researchers worked with different data demonstrated the value that all forms of data count. For example, existing monitoring data might be repurposed to answer an evaluation question on implementation fidelity across regions. New studies could collect new empirical data on norms-shifting which had not yet been systematically collected or examined, or experiential data could be drawn from insider knowledge held by NSI staff, stakeholders and participants (it engages with the perceptions, experiences, reflections, and observations of staff, stakeholders and community members who facilitate or participate in interventions). Discussions of data and evidence led to sharing different interpretations and understandings of how programmes worked allowing for consensus and additional questions to emerge.
By incorporating realist values, it was easier to discern a project’s effects that can be hidden in the perceptions, opinions and implicit or explicit logics of the community actors which underlie their attitudes and actions vis-à-vis a projects’ resources. The embedding of values into the realist evaluation process allowed for an emic understanding of a programme operating within its social context (Lacouture et al. 2015; Pawson & Tilley 1997).
Interrelated effects and increased programme-evaluation significance
To close this section, the authors share three interrelated, at times surprising, effects of their realist evaluation experience that foster substantive evaluation of SBC, with examples to illustrate their points: (1) valuing insider knowledge and critical reflection of context increases programme and evaluation significance; (2)participatory TOC-building connects programmes, implementation context and evaluation research in profound ways that benefit programmes and evaluations; and (3) accepting a range of data types broadens the value of data to inform decision-making by more people.
Valuing insider knowledge and critical reflection of context increases programme and evaluation significance
Norms-shifting interventions tug at the normative realities in communities to create enabling environments that facilitate behaviour change. Bringing diverse stakeholder groups early into an evaluation with more diverse data and measurement points reaffirms the values inherent of locally led evaluation approaches, which aim to foster local critical reflection. A participatory and straightforward theory development process revisited over time and performed in situ brings evaluation into projects early and grounds the process. While evaluation emphasises the importance of determining the value and worth of an intervention, the significance, values and meanings that a group ascribes to a programme are less often discussed. Yet the reactions of multiple stakeholders bring evidence of the ability of realist evaluation to establish significance of an NSI, and by extension, to evaluation (see Box 1).
| BOX 1: Valuing insider knowledge and critical reflection of social contexts. |
Participatory theories of change-building connects programmes, implementation context and evaluation research in new and more profound ways that benefit both parties
The authors’ experience shows that when a diverse group knowledgeable about an NSI refine a TOC to explicitly recognise how normative change occurs, new insights emerge from different stakeholders on how activities lead to seemingly disparate individual and normative outcomes. Partner organisations new to realist evaluation often remarked that the participatory development of a programme theory of change was at times surprising. It unpacked the perceptions and assumptions about a programme and allowed new mechanisms to emerge that were not included in the initial TOC, providing new insights into the context–mechanism–outcome factors and programme effectiveness (Adepoju et al. 2017; Danks 2017; Minyard et al. 2017). Thus, it was beneficial for evaluating NSI projects whose mode of action is not well understood and equally beneficial for NSIs being upscaled (see Box 2).
| BOX 2: Participatory theories of change-building to connect programmes and evaluation research. |
Accepting a range of data types broadens the value of data to inform decision-making by more people
Realist evaluation emphasises using evaluation to answer essential implementation questions and valuing multiple forms of data. Table 2 outlines the range of data types and questions used to address these evidence gaps and serves to demonstrate the value of using different types of data.
| TABLE 2: The range of data types used to answer different questions to address evidence gaps. |
When asking the right research questions derived from agreed-upon information gaps, new evidence was found in all data types. To address evidence gaps identified by Husbands’ School stakeholders (the dashed boxes in Figure 2), for example, the researchers used existing data (staff experience and field observations, project monitoring data, previous studies) and new programme and evaluation research studies (where it was not possible to use existing data). Existing monitoring data were reanalysed and showed that schools were implemented similarly across regions. A quasi-experimental outcome study in pilot and expansion regions showed similar norms-shifting effects were occurring across regions. Experiential data brought in perspectives from multiple insider groups, those who are the closest to communities and understand better the lived realities of communities, and provided confirming evidence that the TOCs were working as expected.
The authors’ experience conducting multiple realist evaluations leads them to believe that a focus on a broader range of data creates greater cross-ownership of findings as different groups ‘own’ and can contribute different data. Attempts to use existing data led to some false starts. In Senegal, efforts were made to collect health and education service data from local authorities to determine if rates of school dropouts and teen pregnancy were less prominent in communities reached by the project compared with nonintervention communities; these efforts were abandoned when data were not easy to retrieve or interpret to answer the evaluation question. Yet even the false starts led to new stakeholder understandings of possibilities where evidence might reside, including in existing service data.
It was found that along the way, M&E and programme staff of the NSIs, understanding new possibilities of data use, proposed new joint activities with Passages evaluators: to simplify monitoring systems to yield helpful information for tracking and understanding bottlenecks (Niger) and to begin more systematic, field-based monitoring of activities and intermediate effects going forward (Senegal). Norms-shifting intervention programme and M&E staff felt like they had a stake in using M&E data for learning and theory confirmation.
Conclusion
Realist evaluation is growing but not yet widespread in Africa and specifically not used much in behavioural science interventions. Yet its application helps decision-makers use a fuller range of programme-generated evidence and experiential knowledge to understand how programmes work to make practice and research-informed decisions and actions. As a programme typology, NSIs are in their nascence. Realist evaluation contributes to local theory development and programme practice and deepens broad stakeholder appreciation of evidence-based adjustments to programmes when done intentionally. In evaluation settings where resources are constrained, it can lead to efficiencies in research through focused inquiry.
The utility of realist evaluation to contribute to knowledge translation and learning is that it expects programme and M&E staff to work together, bound by an exploration of a TOC, as a basis for focused inquiry on evidence gaps in theory. The researchers’ application of realist evaluation demonstrates a way to embed learning and evaluation more concretely into programming by integrating the tenets of realist evaluation with realism values that bring substantive co-creation and use of evidence more fully into practice. Encouraging greater use of realist evaluation with NSIs and other SBC programmes in health and other sectors can expand current evaluation approaches and bring more Africa-centred innovation to the field (Mbava & Chapman 2020). It creates a nexus and a unique and significant dynamic between programme implementers, local stakeholders and evaluators that transcends research and programme practice.
Acknowledgements
The realist evaluations discussed in this article reflect our close intellectual and evaluation collaborations in Niger with Issa Sadou (United Nations Fund for Population Activities [UNFPA] – Niger) and Mohamed Haidara (SongES), the developers of the Husbands’ Schools concept and its male engagement approach, who continue to support school refinement and expansion in Niger and West Africa. In Senegal, our close collaborators’ intellectual and evaluation collaborations include Mamadou Coulibaly and Judi Aubel (Grandmother Project: Change Through Culture), the developers of the Girls Holistic Development (GHD) concept and its grandmother-inclusive approach, who continue to support GHD refinement and expansion.
Competing interests
The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.
Authors’ contributions
S.I. and M.D. led the design partnership for the realist evaluations in Niger and Senegal. S.I., M.D. and A.K. supported in-country evaluations and rapid studies in Niger and Senegal. C.F. conducted the scoping review of realist evaluation. S.I., M.D., A.K. and C.F. led the conception and writing of the first draft of this manuscript. All authors participated in analysis and interpretation of the findings and reviewed draft manuscripts. All authors read and approved the final manuscript.
Funding information
The realist synthesis analysis and article were supported by Passages project funding under the United States Agency for International Development (USAID) Cooperative Agreement (reference number: AID-OAA-A-15-00042).
Data availability
Data sharing is not applicable to this article as no new data were created or analysed in this study.
Disclaimer
The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of USAID or Georgetown University.
References
Adepoju, I., Ilozumba, O., Marchal, B., Zweekhorst, M. & Dieleman, M., 2017, ‘Effects of mHealth for maternal health workers in low and middle-income countries: Findings from a realist review’, in Realist evaluation: Origins and destinations: From promise to practice 20 years on. International Conference for Realist Research, Evaluation and Synthesis, Brisbane, 24–26 October, 2017, p. 35.
Aslam, R., Hendry, M., Booth, A., Carter, B., Charles, J., Craine, N. et al., 2017, ‘Intervention Now to Eliminate Repeat Unintended Pregnancy in Teenagers (INTERUPT): A systematic review of intervention effectiveness and cost-effectiveness, and qualitative and realist synthesis of implementation factors and user engagement’, BMC Medicine 15, 155. https://doi.org/10.1186/s12916-017-0904-7
CORE Group, 2021, Complexity-aware monitoring, evaluation and learning for social and behavior change interventions: Advocacy booklet, viewed 16 September 2022, from https://coregroup.org/resource-library/complexity-aware-monitoring-evaluation-learning-for-social-and-behavior-change-interventions/.
Danks, M., 2017, ‘A complex realist inquiry exploring the implementation of a peer support initiative in the prison setting: Application of key concepts’, in Realist evaluation: Origins and destinations: From promise to practice 20 years on. International Conference for Realist Research, Evaluation and Synthesis, Brisbane, 24–26 October, 2017, p. 5.
Davis, R., Campbell, R., Hildon, Z., Hobbs, L. & Michie, S., 2015, ‘Theories of behaviour and behaviour change across the social and behavioural sciences: A scoping review’, Health Psychology Review 9(3), 323–344. https://doi.org/10.1080/17437199.2014.941722
Dossou, J. & Marchal, B., 2017, Realist evaluation: Origins and destinations: From promise to practice 20 years on. International Conference for Realist Research, Evaluation and Synthesis, p. 7, Brisbane.
Duncan, V., Groot, G., Waldron, T., Carr, T., Roberts, R., McMullen, M. et al., 2017, ‘Supporting Saskatchewan’s Indigenous peoples in making optimal patient-centred health decisions: The development and assessment of realist program theory’, in Realist evaluation: Origins and destinations: From promise to practice 20 years on. International Conference for Realist Research, Evaluation and Synthesis, Brisbane, 24–26 October, 2017, p. 3.
Jagosh, J. & Tilley, N., 2016, ‘Realist evaluation at 25: Cumulating knowledge, advancing debates and innovating methods’, Evaluation 22(3), 267–269. https://doi.org/10.1177/1356389016656502
Lacouture, A., Breton, E., Guichard, A. & Ridde, V., 2015, ‘The concept of mechanism from a realist approach: A scoping review to facilitate its operationalization in public health program evaluation’, Implementation Science 10, 153. https://doi.org/10.1186/s13012-015-0345-7
Liu, H., 2017, Realist evaluation: Origins and destinations: From promise to practice 20 years on. International Conference for Realist Research, Evaluation and Synthesis, p. 2, Brisbane.
Marchal, B., 2017, ‘Realist approaches in international development’, in Realist evaluation: Origins and destinations: From promise to practice 20 years on. International Conference for Realist Research, Evaluation and Synthesis, Brisbane, 24–26 October, 2017, p. 6.
Mathias, K., Mathias, J., Goicolea, I. & Kermode, M., 2018, ‘Strengthening community mental health competence – A realist informed case study from Dehradun, North India’, Health and Social Care in the Community 26(1), e179–e190. https://doi.org/10.1111/hsc.12498
Mbava, N. & Chapman, S., 2020, ‘Adapting realist evaluation for made in Africa evaluation criteria’, African Evaluation Journal 8(1), a508. https://doi.org/10.4102/aej.v8i1.508
Minyard, K., Smith, T., Turner, R. & Milstein, B., 2017, ‘Realist evaluation identifies community and programmatic factors influencing effective use of system dynamic models’, in Realist evaluation: Origins and destinations: From promise to practice 20 years on. International Conference for Realist Research, Evaluation and Synthesis, Brisbane, 24–26 October, 2017, p. 5.
Nguyen, G., Costenbader, E., Plourde, K., Kerner, B. & Igras, S., 2019, ‘Scaling-up normative change interventions for adolescent and youth reproductive health: An examination of the evidence’, Journal of Adolescent Health 64(4S), S16–S30. https://doi.org/10.1016/j.jadohealth.2019.01.004
Pawson, R.T. & Tilley, N., 1997, Realist evaluation, Sage Publications, London.
Rayment-McHugh, S., 2017, ‘A comprehensive model for preventing sexual violence and abuse in two Australian Indigenous communities: A realist evaluation’, in Realist evaluation: Origins and destinations: From promise to practice 20 years on. International Conference for Realist Research, Evaluation and Synthesis, Brisbane, 24–26 October, 2017, p. 10.
Robert, E. & Ridde, V., 2013, ‘L’approche réaliste pour l’évaluation de programmes et la revue systématique: De la théorie à la pratique’, Mesure et évaluation en éducation 36(3), 79. https://doi.org/10.7202/1025741ar
Salter, K. & Kothari, A., 2014, ‘Using realist evaluation to open the black box of knowledge translation: A state-of-the-art review’, Implementation Science 9, 115. https://doi.org/10.1186/s13012-014-0115-y
The Learning Collaborative to Advance Normative Change, 2017, Identifying and describing approaches and attributes of norms-shifting interventions, Institute for Reproductive Health, Georgetown University, Washington, DC.
The Learning Collaborative to Advance Normative Change, 2019, Social norms and AYSRH: Building a bridge from theory to program design, Institute for Reproductive Health, Georgetown University, Washington, DC.
Tilley, N., 2017, ‘Realist evaluation: Origins and destinations’, in Realist evaluation: Origins and destinations: From promise to practice 20 years on. International Conference for Realist Research, Evaluation and Synthesis, Brisbane, 24–26 October, 2017, p. 1.
Vugts, M., Joosen, M. & Vrijhoef, H., 2017, ‘When, for whom, and how serious gaming outcomes are achieved by patients during multidisciplinary rehabilitation; a realist process evaluation’, in Realist evaluation: Origins and destinations: From promise to practice 20 years on. International Conference for Realist Research, Evaluation and Synthesis, Brisbane, 24–26 October, 2017, p. 2.
Westhorp, G., 2014, Realist impact evaluation: An introduction, London, viewed 16 September 2022, from https://cdn.odi.org/media/documents/9138.pdf.
|
Hide
Show all
Email this article (Login required)
