Programme evaluation has followed different trajectories across regions. In Africa, its expansion has been driven by non-governmental organisations and the private sector (Basheka 2015), though governments increasingly recognise its value (Abrahams 2019). While the practice grows in relevance across sectors, evaluation is often conflated with monitoring, and capacity gaps persist in national monitoring and evaluation (M&E) systems (Abrahams 2015, 2019). Non-governmental organisations and civil society implement numerous projects but often struggle to meet stakeholders’ expectations for rigorous evaluation (Chaplowe & Engo-Tega 2007). Many practitioners also lack formal training in evaluation studies (Agonnoude et al. 2021). Although academic and non-academic training initiatives are growing, offerings remain limited, with only 13 of 26 South African universities offering such programmes (Mouton, Wildschut & Leslie 2018). Regional voluntary organisations for professional evaluation (VOPEs) continue to play a key role in fostering evaluation capacity.

Ensuring evaluation integrity is a global and continental priority (Agonnoude et al. 2021). Professional guidelines emphasise ethical and technical standards. The American Evaluation Association highlights systematic inquiry and competence (Rallis, Rossman & Gajda 2007). Similarly, the African Evaluation Association’s (AfrEA) Evaluation Principles (2021) promote evaluations that are trustworthy, ethical and contextually grounded. Principle 2 underscores the need for rigour, transparency, reflexivity and feasibility, while Principle 3 stresses ethics and equity (AfrEA 2021). These guidelines align with debates on quality standards for qualitative approaches, increasingly relevant in evaluation. Qualitative methods provide contextual insights into how and why interventions work (Spencer et al. 2003). In evaluation, quantitative approaches focus on measuring outcomes and impacts, while qualitative approaches offer interpretive insight into these processes.

Lincoln and Guba (1985) introduced ‘trustworthiness’ as the qualitative counterpart to reliability and validity in the quantitative tradition. They proposed four criteria: credibility, transferability, dependability and confirmability – each linked to specific techniques. Since Lincoln and Guba’s (1985) work, perspectives on qualitative rigour have evolved. While some advocate for applying traditional criteria to qualitative work (Creswell & Miller 2000), others argue for distinct standards aligned with qualitative epistemologies (Curtin & Fossey 2007; Novak & Jen 2024). Consequently, multiple frameworks exist, though consensus remains on the need for rigorous reporting (Hammersley 2007).

These debates are not merely academic: in the African evaluation context, where the field is still professionalising, practitioner training is uneven, and evaluation findings directly inform resource allocation and development decisions (Agonnoude et al. 2021; Tarsilla 2014), how trustworthiness is established and communicated carries particular practical weight. Evaluators operating within complex accountability environments – including donor-driven pressures, stakeholder expectations and internal learning demands (Tirivanhu 2022) – often have limited institutional support for methodological reflection (Agonnoude et al. 2021). Clear standards for demonstrating rigour are therefore necessary, yet the proliferation of competing frameworks (Dixon-Woods et al. 2004) risks obscuring rather than guiding practice. Continental frameworks such as AfrEA’s Evaluation Principles (2021), which explicitly foreground rigour, transparency and reflexivity as constitutive of evaluation quality, signal that the African evaluation community recognises this need. The question this study addresses is whether that recognition is reflected in how qualitative evaluations are reported.

Discussions on the quality and trustworthiness of qualitative inquiry have influenced fields including education (Liao & Hitchcock 2018), social work (Enworo 2023), health sciences (Santiago-Delefosse et al. 2016) and programme evaluation (Spencer et al. 2003). Still, empirical work assessing how trustworthiness is demonstrated in practice remains limited. Liao and Hitchcock (2018) analysed 118 higher education evaluations, identifying commonly used trustworthiness techniques and offering a reporting framework that informs this study. We focus specifically on peer-reviewed articles reporting on qualitative components of evaluations in African contexts.

Three studies provide a foundation for understanding evaluation reporting in Africa. Ndhlovu et al. (2019) analysed 338 reports in the African Evaluation Database (AfrED) (2005–2016), finding high use of qualitative (55.60%) and mixed methods (35.80%) but limited methodological detail. Ngwabi and Wildschut (2019) reviewed 142 donor-commissioned reports, finding low adherence to Organisation for Economic Co-operation and Development (OECD) standards, especially in sampling and methodology. They also highlighted the underuse of African evaluators. Chirau, Tirivanhu and Ramasobana (2019) examined 20 qualitative reports using an 18-item checklist. Most were rated medium quality, with major gaps in reflexivity, ethics and methodological alignment.

Together, these studies highlight persistent gaps in evaluation reporting. However, they focus on grey literature. We argue that peer-reviewed publications, guided by journal standards and a review process, offer stronger insights into reporting practices, where consistent rigour is expected (McWilliam 2000; Novak & Jen 2024). Additionally, evaluators rely on academic research to guide practice: a survey of American Evaluation Association members found 80.95% regularly read published research (Coryn et al. 2016). This study examines how African evaluation authors respond to reporting expectations, offering lessons for improving practice.

To guide the inquiry, we asked:

Guided by the PCC framework, we searched the WoS database in July 2022 using the query:

TS = (qualitative research AND program* AND evaluation) AND CU = (Africa).

The search terms were derived from the core components of the review: qualitative evaluation approaches (Population), trustworthiness and methodological practices (Concept) and African study contexts (Context). Web of Science was selected for its extensive coverage of peer-reviewed literature, structured indexing and suitability for systematic searching. We note that in WoS, the CU field restricts searches to the country or region. CU = ‘Africa’ was therefore used as a broad retrieval parameter to capture studies indexed as relating to African contexts. While this approach supports exploratory coverage, it may not fully capture all country-specific records and therefore represents a limitation of the search strategy.

Consistent with the exploratory nature of scoping reviews, the search aimed to identify a broad yet manageable corpus rather than achieve exhaustive coverage. We note that the resulting dataset represents WoS-indexed, peer-reviewed evaluation studies conducted in African contexts, rather than the full universe of African evaluation practice. We acknowledge that additional sources, including the AfrED, may yield further studies. However, our focus on WoS-indexed publications reflects the study’s exploratory objective of examining reporting practices within academically published evaluation research, complementing prior AfrED-based and institutional evaluation studies (Chirau et al. 2019; Ndhlovu et al. 2019; Ngwabi & Wildschut 2019).

The search yielded 189 studies. After applying inclusion criteria (evaluation studies using qualitative methods as stand-alone or mixed and focused on Africa), 83 publications were retained. We excluded duplicates, studies not based on African contexts, non-empirical papers and those lacking qualitative data. While the search does not claim exhaustive coverage, the systematically derived sample is sufficient to support the study’s objective of mapping trustworthiness reporting practices. All included studies are listed in Online Appendix 1: Table 1. The review was conducted in line with established guidance for scoping reviews (Peters et al. 2020), with methodological adaptations appropriate to its exploratory scope and analytical focus. A pre-registered protocol was not developed and full compliance with reporting standards such as Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) is not claimed; rather, scoping review principles were selectively applied to suit the study’s objective.

To analyse how trustworthiness is reported in qualitative evaluation studies, we developed an analytical framework based on established literature on qualitative rigour and trustworthiness (e.g. Morse 2015; Spencer et al. 2003). The framework draws primarily on Liao and Hitchcock (2018), who identify key techniques used to establish trustworthiness in qualitative evaluation. As summarised in Table 1, these sources collectively provide a comprehensive foundation for identifying and categorising trustworthiness strategies.

As originally presented in Liao and Hitchcock (2018), trustworthiness strategies were organised into two categories: primary design techniques and additional trustworthiness techniques. The former represents techniques that are fundamental to the design phase and corresponding to the structurally visible elements of research reports, including research design, sampling, data collection, data analysis and thick description. The latter techniques comprise supplementary strategies whose application varies depending on the nature of the research, including triangulation, audit trails, member checking, peer debriefing, negative case analysis and others (see Table 1), and which often require deeper analytic or relational accountability.

This approach shifts focus from classic categories such as credibility, transferability, dependability and confirmability (Lincoln & Guba 1985; Novak & Jen 2024), and associated debates about their parallels with quantitative quality criteria (Morse 2015), to observable practices through which trustworthiness is enacted and reported in published studies (Curtin & Fossey 2007). For analytical purposes, all strategies were treated as conceptually relevant but not hierarchically weighted, as the study aims to map reporting patterns rather than evaluate the relative importance of individual techniques.

Our adaptation further incorporated additional techniques that emerged during the coding process, extending Liao and Hitchcock’s (2018) framework to reflect how trustworthiness is operationalised in the reviewed studies. These include ethical considerations under primary design techniques and inter-coder agreement and coding system description under additional trustworthiness techniques. As Lub (2015) and AfrEA (2021) emphasise, ethical considerations are crucial given the relational dynamics. Similarly, when multiple investigators are involved, inter-coder agreement and clear coding system descriptions support research rigour.

Additionally, we recorded descriptive characteristics for each publication: year of publication, author’s country of affiliation, evaluation site (country), thematic focus area (e.g. health, education, agriculture), qualitative data type (e.g. interviews, observations) and use of CAQDAS to characterise methodological breadth.

Three document variables – methods, focus area and CAQDAS use – were used to explore variations in trustworthiness reporting through MCA, a method suited to revealing patterns in categorical data (Kienstra & Van der Heijden 2015). Multiple correspondence analysis enables the exploration of multiple categorical associations simultaneously, making it appropriate for our purposes (details provided in a later section). Computer-Assisted Qualitative Data Analysis Software use has received particular attention due to its relevance in supporting analytic rigour and transparency (Kapiszewski & Karcher 2021). Based on experience using and teaching ATLAS.ti, we expected CAQDAS-supported studies to show stronger trustworthiness reporting.

Figure 1 outlines our analytic workflow for the 83 publications using ATLAS.ti. Following the directed qualitative content analysis method (Hsieh & Shannon 2005), we applied a structured framework developed from the literature, complemented by inductive refinement and both latent (interpretive) and manifest (descriptive) coding. Directed content analysis enables systematic classification of textual data to identify patterns. Our approach blended predefined concepts with data-driven interpretation, allowing iterative refinement of the initial coding framework to incorporate emergent categories (e.g. ethics, inter-coder agreement).

We used a two-level coding strategy:

For instance, a study’s research design was coded as present if it included any of the following: description, justification, alignment with purpose or relevance to research questions. A subsequent step involved identifying and sub-coding specific elements (e.g. design type, justification, stated purpose, and alignment with research questions). This detailed breakdown was applied across all techniques where granularity was necessary. The second-level analysis determined whether a technique was comprehensively or partially reported, based on the sub-codes (see the sub-code indicators in Online Appendix 1: Table 2, Table 3 and Table 4) illustrated above. Comprehensive reporting was recorded when all sub-category indicators were present for a strategy and partial when one or more elements were missing. Final classification was conducted in Excel, using ATLAS.ti’s Code-Document Table outputs. The coding structure is fully documented in the accompanying Online Appendix 1.

Our results include descriptive statistics and MCA. Firstly, we present frequencies and percentages summarising the use of trustworthiness techniques across documents. Secondly, we used MCA to explore how strategies varied by methodology, thematic focus and CAQDAS use.

Two key functions informed the MCA analysis: the eigenvalue, which indicates the variance (inertia) explained by each dimension, and the squared cosine (Cos²), which assesses the quality of a category’s representation in a dimension (Kienstra & Van der Heijden 2015). Eigenvalues help identify the most influential dimensions. Dimensions 1 and 2, explaining 8.86% and 7.33% of the variance, respectively (16.19% total), were retained due to their interpretive strength (see Figure 2). While these dimensions explained a modest proportion of variance, they were prioritised due to their interpretive relevance, consistent with exploratory analysis where low-dimensional solutions capture meaningful patterns (Husson, Le & Pagès 2017). A Cos² of 0.1 threshold was used to exclude weakly represented categories and minimise overinterpretation, consistent with exploratory multivariate analysis practices that retain interpretively meaningful associations (Husson et al. 2017).

Multiple correspondence analysis was conducted in R using Factoshiny and FactoMineR. The dataset comprised 26 variables (three document-level, 23 trustworthiness strategies) across 83 studies. Document features were inputted as supplementary variables, and trustworthiness strategies as active. Results are presented as a biplot and summary table, with full statistical results available in the accompanying Online Appendix 2 (Excel file).

To strengthen trustworthiness, we piloted the review using the full dataset and presented initial findings at the World Conference on Qualitative Research (January 2024). Feedback informed the shift to a full-text review. Author 1 conducted all coding and analysis, while Author 2 independently reviewed coded segments, coding decisions and category definitions. Disagreements were discussed and resolved through iterative comparison and refinement of the coding framework, supported by analytic memos and an audit trail documenting coding decisions. Given our interpretive approach, formal inter-coder agreement metrics were not applied; instead, consistency was enhanced through systematic verification and transparent documentation.

We also acknowledge the researchers’ positionality in shaping the analysis. Both authors have experience in qualitative methods and CAQDAS (including ATLAS.ti), which informed initial expectations regarding analytic rigour and its transparent reporting. These assumptions were critically reflected upon during the analysis, particularly where findings did not align with expectations.

The accompanying Online Appendix 1 provides a detailed account of the dataset and analytic procedures, including the bibliography of the 83 studies and the coding structure, while Online Appendix 2 (Excel file) provides the structured dataset exported from ATLAS.ti and the MCA outputs (see Online Appendix 1: Table 5, for brief descriptions of the nine spreadsheets). We also maintained an audit trail, provided thick description in this report and conducted peer debriefing (early findings were shared with colleagues for feedback), reflecting established trustworthiness strategies (Curtin & Fossey 2007; Novak & Jen 2024).

This study relied solely on published literature and did not involve human participants or animals. An ethics-exempt approval letter was obtained from Stellenbosch University’s Research Ethics Committee: Social Behavioural and Education Research (REC: SBE). The study was conducted in accordance with recognised ethical standards for research, including the principles outlined in the Singapore Statement on Research Integrity (2010).

Our analysis covered 83 qualitative evaluation studies conducted in Africa, grouped by methodological orientation, thematic focus and CAQDAS use. As shown in Figure 3, more than half of the articles (63.9%) used qualitative methods, while others (36.1%) adopted mixed methods, combining qualitative and quantitative data. Interviews were the predominant data source (77.1%), frequently supported by focus group discussions (34.9%), reflecting participatory tendencies (see Figure 3).

Evaluations spanned seven thematic domains, with health (50.6%) and education (27.7%) dominating the corpus, while sectors like business (two studies) and sports (three studies) were underrepresented (Figure 3). Geographically, the corpus was heavily concentrated in South Africa, which accounted for 72 evaluations, with 90% of authors (n = 286) affiliated with South African institutions. In contrast, only a small number of studies were conducted in eSwatini (four studies) or involved cross-national collaborations (five studies) (Figure 4). This concentration reflects South Africa’s established prominence in African evaluation scholarship (Erasmus, Jordaan & Stewart 2020; Mouton et al. 2018). These findings should therefore be interpreted with caution, as the dominance of South African cases may limit broader generalisation across Africa.

Overall, the data highlight growing use of qualitative approaches in African evaluation, both as standalone methods and in combination with quantitative methods. Notably, 41% of studies employed CAQDAS, particularly NVivo (18 studies) and ATLAS.ti (14 studies), underscoring the growing role of digital tools in supporting qualitative workflows. No use of artificial intelligence (AI)-assisted qualitative analysis tools was identified, consistent with the July 2022 data collection timeframe, prior to their wider adoption.

The analysis identified 23 trustworthiness strategies, categorised as primary design techniques (fundamental to methodological rigour) and additional trustworthiness techniques (supporting interpretive accountability). Table 2 presents these strategies by comprehensiveness of reporting. The findings reveal substantial variability in reporting practices, with strong adherence to foundational strategies but notable gaps in the reporting of an additional category.

Reporting on primary design techniques was strongest where strategies are foundational and characteristic of published text – notably thick description, data collection procedures and study design – but weakened considerably where deeper methodological reflection was required. Thick description (89.2%) and data collection (86.8%) were the most comprehensively reported, while ethical considerations, though addressed in over 60% of studies, were absent in more than a quarter.

By contrast, the strategies most critical to methodological accountability – analytic details and sampling – were the least comprehensively reported. Mention was common, but substantive explanation was rare: while 83.1% of studies referenced some form of analysis, only 13.3% offered detailed accounts of coding approaches, analytic progression or verification procedure. The gap between presence and depth was equally stark for sampling, which appeared in 90.4% of studies but was comprehensively described in only 9.6% – with strategy justification, sample size rationale and recruitment procedures rarely elaborated. One study’s account that ‘themes emerged through open coding’, without clarifying how codes were consolidated or validated, illustrates a broader pattern: analytic processes were acknowledged but not made transparent enough for independent scrutiny. These omissions are consequential as they affect assessments of transferability and rigour.

Among additional techniques, uptake followed a clear pattern: strategies that leave visible traces in a research report were well-represented, while those requiring sustained relational or analytic investment were largely absent. Triangulation was the most consistently applied (92.8%), typically through integration of data sources (67 studies) or methods (62 studies). For example, some evaluations combined interviews, focus groups and documents, while others paired qualitative case studies with surveys. Multivocality (89.2%) and comparison and contrast (69.9%) were similarly strong, reflecting a tendency to incorporate diverse voices and perspectives within the written account of findings.

The contrast with participant-centred engagement strategies is striking. Member checking – one of the most direct mechanisms for validating data and interpretations with those who provided the data – was fully reported in only 21.7% of studies and absent in nearly three quarters. Prolonged engagement and persistent observation were almost entirely missing (1.2%). These underuses are striking, given that nearly all studies involved direct human interaction. This indicates preferences for strategies that leave visible textual traces over those requiring documented relational engagement.

Analytic transparency was the most consistently underdeveloped dimension of trustworthiness reporting under the additional techniques category. Audit trails, codebook descriptions, peer debriefing and engagement with rival explanations were rarely evidenced, suggesting limited documentation of analytic decision-making processes. Notably, over half of studies involving multiple coders offered no account of how coding discrepancies were resolved undermining confidence in the consistency of analytic procedures.

In Figure 5, our MCA results focus on categories with cos² ≥ 0.1. The MCA reveals that variation in trustworthiness reporting is patterned rather than random, structured by sectoral context and the presence or absence of digital analytical tools. CAQDAS use (cos² = 0.12 for Yes; 0.12 for No) and agriculture and environment focus (cos² = 0.10) were the only document characteristics with statistically significant contributions to the dimensions (cos² ≥ 0.1), while other characteristics – thematic focus and methodological orientation – informed broader thematic interpretation without reaching statistical significance.

Three clusters emerge from the structure (summarised in Table 3). The first, defined by the positive pole of Dimension 1, groups studies with the most comprehensive reporting profiles, particularly in sampling, analytic detail, ethics and audit trails. Health evaluations and standalone qualitative studies are thematically proximate to this cluster, though their weak statistical loadings (cos² < 0.1) indicate that sectoral affiliation does not reliably predict rigour. Comprehensive reporting in this cluster appears to reflect deliberate methodological commitment rather than structural characteristics.

The second cluster, organised along Dimension 2, is characterised by partial rather than absent strategies, notably member checking and prolonged engagement. Computer-Assisted Qualitative Data Analysis Software use (cos² = 0.12) is the only statistically significant characteristic associated with this cluster. Rather than aligning with analytic accountability mechanisms (e.g. audit trails, reflexivity, codebook development or inter-coder agreement), software use is associated with partial reporting of engagement strategies. This suggests that CAQDAS may support the organisation and documentation of analytic work, and this potential is not reflected in the reporting of analytic processes, nor does it extend to strengthening relational and field-based practices central to trustworthiness.

The third cluster, anchored at the negative pole of Dimension 1, presents the starkest pattern. Agriculture and environment evaluations are statistically linked to the absence of multivocality, ethical reporting and inter-coder agreement, and to only partial thick description – a concentration of omissions indicative of sector-specific capacity constraints. While CAQDAS-No appears (–0.20, 0.05) in proximity to this cluster, it is more strongly associated with Dimension 2, indicating contextual co-occurrence rather than a direct causal role.

Together, the clusters suggest trustworthiness reporting in the WoS-indexed evaluation studies analysed is shaped less by methodological orientation than by sectoral and technological contexts, with agriculture and environment emerging as areas of particular concern.

Several caveats should be considered when interpreting the findings of this study. While our analysis focuses on qualitative evaluation research in African contexts, it is limited to WoS-indexed, peer-reviewed publications, excluding unpublished and internal organisational evaluation reports that may provide additional insights. This study complements previous investigations into these other areas in the African context (Ndhlovu et al. 2019; Ngwabi & Wildschut 2019). The focus on WoS-indexed and English-language publications may also exclude relevant studies, particularly from non-English-speaking contexts, with implications for geographic representation. All coding was conducted by a single coder. While several trustworthiness strategies were applied, coding decisions inevitably reflect interpretive judgement, as is standard in qualitative research.

Additionally, our analysis is based on reported methodological practices, meaning actual evaluation practices may differ from what is documented. The findings should therefore be interpreted as indicative of reporting patterns rather than definitive assessments of practice or quality. The study also does not assess the substantive findings of the evaluations themselves.

Finally, consistent with its exploratory design, MCA was employed to identify patterns in the data. The results therefore reflect associations rather than causal relationships. Future research could benefit from larger and more exhaustive samples that explore causal links between methodological rigour and factors such as donor influence, commissioner-evaluator relationships and institutional contexts. Future research could extend this work using larger and more diverse samples to examine how trustworthiness relates to factors such as donor influence, commissioner–evaluator relationships and institutional contexts.