Original Research

Exploring a new tool for assessing and enhancing climate resilience in smallholder farm systems

Bantu B. Mabaso

Received: 30 Aug. 2025; Accepted: 18 Dec. 2025; Published: 10 Apr. 2026

Copyright: © 2026. The Author Licensee: AOSIS.
This work is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license (https://creativecommons.org/licenses/by/4.0/).

Abstract

Background: Smallholder farmers in sub-Saharan Africa are at the forefront of climate change impacts, facing significant challenges to their livelihoods because of increasing temperatures, erratic rainfall and extreme weather events. While numerous tools have been developed to assess climate resilience interventions, their effectiveness is often difficult to ascertain, leaving development practitioners uncertain about the true impact of their work.

Objectives: This article aims to address this challenge by examining the application of a newly developed tool, the Farm Resilience Assessment Scorecard (FRAS). The primary objective is to highlight critical lessons from its implementation to inform and improve evaluation practice in the context of climate resilience.

Method: An exploratory approach was employed to analyse the application of the FRAS tool in smallholder farm systems. This involved a critical reflection on the tool’s implementation process, focusing on its practical utility, and its ability to capture the multidimensional nature of resilience.

Results: The article emphasises the need for mixed-method approaches and participatory engagement to ensure that resilience assessment tools are not merely extractive but are genuinely empowering for smallholder farmers.

Conclusion: The FRAS serves as a viable, low-burden tool for quantifying climate resilience in resource-constrained environments. However, to move beyond extractive data collection, the FRAS is most effective when implemented through the participatory, mixed-method lens identified in this study. By integrating its quantitative simplicity into broader evaluation systems, practitioners can empower farmers while ensuring the multi-dimensional nature of resilience is qualitatively captured, ultimately leading to more responsive and context-specific climate interventions.

Contribution: This article contributes practical lessons on the application of a novel resilience assessment tool. It offers valuable guidance for evaluation practitioners in designing and conducting more effective and empowering assessments and provides clear recommendations for development practitioners aiming to create more robust and user-centric tools for measuring climate resilience in smallholder agriculture.

Keywords: climate resilience assessment; smallholder farmers; sub-Saharan Africa; participatory methods; evaluation practice; agricultural development.

Introduction

The African continent is perceived to be the region where most farming livelihoods are vulnerable to climate change (Food and Agriculture Organisation 2021; Intergovernmental Panel on Climate Change 2022). This is because agriculture is still critical for economic growth in Africa and is central to the livelihoods of a significant portion of the continent’s population. Of the 33 million farms in Africa, 80% belong to smallholder farmers who produce on less than 2 ha of land and collectively contribute 75% of all agricultural outputs in the region (Hlophe-Ginindza, Nhamo & Mpandeli 2023). These farmers are increasingly exposed to climate-related shocks and stresses such as droughts, floods and erratic rainfall patterns. The intensification of climate change impacts on smallholder farm systems poses a significant threat to Africa’s food security. In response, there has been an increase in development interventions aimed at enhancing the resilience of smallholder farm systems.

Consequently, there has also been a diverse and rapidly expanding set of resilience assessment tools and frameworks that are employed by a wide spectrum of actors – including international development organisations, national governments and academic researchers – to inform the design, implementation, monitoring and evaluation of interventions that aim to improve the resilience of smallholder farmers. These tools are crucial because the data and results they generate are increasingly used to make strategic decisions on how and where to scale the right interventions to amplify impact and direct climate finance as well as development funding.

However, there is a disconnect that persists between the complex, multidimensional theory of resilience and its practical application and measurement in the field, which can make it challenging to determine from assessments if efforts to improve resilience are indeed working. Most of the literature that focuses on developing resilience measurement frameworks is limited in that it is often unclear how the frameworks link theory to practice so that the guidance they provide for measuring resilience can translate into operational tools that can advance development outcomes (Dixon & Stringer 2015). This means that many assessments are undertaken without explicit consideration for the rich body of resilience thinking in which they should be grounded, making it difficult to understand how conceptual nuances are translating into practice and whether the tools in use are truly capturing the essence of what resilience means to smallholder farming communities.

Tools developed to assess climate resilience in smallholder farm systems range from complex, data-intensive assessment tools to more simplified, user-friendly tools like scorecards. Each of these tools has unique strengths and weaknesses, but a common challenge among them is reconciling the trade-off between rigour and feasibility. As Macheka et al. (2022:203) argue in their review of resilience measurement in Zimbabwe, tools that are too complex and data-intensive are often impractical for use, especially in resource-constrained settings like sub-Saharan Africa. While oversimplified tools can be easier to use, they also run the risk of failing to capture the nuances of resilience, resulting in misleading conclusions. This article contributes to the growing body of literature on climate resilience assessment by providing a critical analysis of the Farm Resilience Assessment Scorecard (FRAS), which attempts to strike a balance between these two extremes.

The objectives of this article are twofold. The article presents the process of developing and applying the FRAS, drawing on a dissertation project that sought to quantify the climate resilience of smallholder farm systems in Malawi. The article also critically reflects on the findings from this process, with a focus on what has been learned about the practicalities of assessing climate resilience in smallholder farm systems using the FRAS. The central argument of this article is that the FRAS offers various benefits for the monitoring and evaluation of climate resilience by: (1) enabling easy data collection for climate resilience assessment, empowering even farmers to conduct evaluations without needing a specialist, (2) focusing on a few key indicators that can be frequently and quickly monitored, avoiding lengthy paper surveys while also allowing for faster identification of crucial intervention points to boost resilience and (3) enabling longitudinal studies of farm trajectories over time by supporting continuous data collection to better illuminate the nuances of resilience, which only emerge with time. Even so, the FRAS’ utility can be maximised when it is used as part of a broader, more participatory monitoring and evaluation system. By sharing lessons learned from the application of the FRAS and exploring the implications these lessons have for evaluation practice and the design of similar tools, this article aims to provide valuable insights for evaluators, development practitioners and researchers working to support smallholder farmers in their efforts to build a more climate-resilient future for their communities.

Literature review: The assessment of climate resilience in African smallholder farm systems

Defining resilience in the context of farm systems

The effective assessment of climate resilience requires establishing a clear and operational definition of the concept. Resilience theory structures analysis around three distinct but interrelated capacities:

• Absorptive capacity: This is the ability of a system to cope with and recover from immediate, short-term shocks and stresses using its existing resources and strategies (Lutta et al. 2024). It is the capacity for persistence and ‘bouncing back’. Ifejika Speranza (2013:24) refers to absorptive capacity as buffer capacity and states that the ability to ‘buffer small, incremental shocks over a longer period is very important as it determines the ability to withstand greater shocks through improved learning from past experiences’. For a smallholder farmer, absorptive capacity includes preventative measures and coping mechanisms such as relying on personal savings to buy food after a poor harvest, selling small livestock, reducing consumption or accessing informal credit from community members (Teklu, Simane & Bezabih 2023). This capacity is essential for immediate survival.
• Adaptive capacity: This is the ability of a system to adjust to external drivers of change and is another key feature that can support the persistence of a system’s primary functions and structures. It refers to the ability to learn, innovate and make proactive choices to adjust livelihood strategies in response to medium- to long-term changes and to take advantage of new opportunities that arise (Lutta et al. 2024). It involves incremental changes and adjustments to the existing system. Examples from African smallholder contexts include adopting drought-tolerant seed varieties, implementing soil and water conservation techniques, diversifying crops and livestock or introducing small-scale irrigation systems (Shilomboleni, Epstein & Mansingh 2024).
• Transformative capacity: In some instances, shocks or disturbances to the system might be large enough that important functions begin to fail. For a farm, these could be food production or income generation. This requires a fundamental transformation as well as innovation. This is where transformative capacity comes in, as it is ‘the creation of a new system when ecological, social and economic conditions flip a system into an undesirable state’ (Walker et al. 2006:13). This is the most profound capacity, involving the ability to make fundamental, systemic changes that alter the core structures and rules of the system when the existing livelihood is no longer viable. This type of capacity becomes necessary when the magnitude of change surpasses the limits of adaptive capacity. Examples include shifting from cropping to pastoralism in response to aridification, replanting a farm with entirely new crops better suited to a new climate and linked to new markets or shifting from raising cattle to establishing a centre for ecotourism to restore land after overgrazing (Cumming et al. 2005).

Climate resilience is not a static state of being, but it is a dynamic set of these capacities, which allow a system – in this case, a farm – to continue to function in the face of a disturbance or disruption (Lutta et al. 2024). The function of importance in this case is the production of agricultural goods and services either for subsistence or to earn an income. A climate-resilient farm system is one that can avoid, absorb or adapt and transform in response to shocks and stresses, thereby maintaining its fundamental identity and functions or reorganising to a new, more viable state (Madin & Bamfo 2021). This means that in the event of a shock to the farm system, a resilient farm would be able to anticipate this disruption, absorb it, adapt to it or completely transform in such a way that the ecosystem services responsible for the provisioning of agricultural goods like food are still maintained.

In some instances, there can be trade-offs and tensions between the three capacities that can be difficult to contend with, meaning that there is always the danger of what Walker et al. (2006:13) refer to as a ‘lock-in’ effect. An example of this is the mining away of nutrients from soil as a farm tries to maintain production under stressed conditions when the farm can no longer produce within its local ecological carrying capacity, eventually making the system impoverished or resource-poor (Allison & Hobbs 2004). The system becomes resilient, but in a negative way that is often detrimental to the environment and human well-being, and this is called a ‘lock-in’ effect because a return to the old state of productivity becomes highly unlikely. This occurs because the farm system can no longer withstand or persist through climate-related shocks and ‘flips’ to an undesirable state or regime where essential ecosystem services can no longer sustain a farm’s core function of providing agricultural goods. This demonstrates that resilience is not always a positive attribute. It is in these instances where transformative capacity becomes important. This is why focusing climate action efforts on building positive resilience, where smallholder farm systems stay in desirable states or regimes in which they can still maintain productivity and support the well-being of humans, is important. Supporting smallholder farmers by preventing their farm systems from ‘flipping’ into undesirable states or regimes that fail to sustain human life in the face of climate change is pertinent to achieving crucial sustainable development goals like the elimination of poverty and hunger, especially in places in the global south like sub-Saharan Africa.

Overarching climate resilience assessment frameworks

In the African context, climate resilience assessment is often framed within broader development approaches, most notably climate-smart agriculture (CSA) and the sustainable livelihoods framework:

• Climate-smart agriculture (CSA): Promoted extensively by the Food and Agriculture Organization (FAO), the World Bank and other major development partners, CSA is an approach that seeks to achieve three integrated objectives, often termed the ‘triple-win’: (1) sustainably increasing agricultural productivity and incomes; (2) adapting and building resilience to climate change and (3) reducing or removing greenhouse gas emissions where possible (World Bank 2017:1). Many interventions in Africa are designed and evaluated against these three pillars. However, critical scholarship, including work by African researchers, points to a significant tension in its application. In practice, many CSA initiatives tend to prioritise the first pillar (productivity) and focus on technical, biophysical solutions like technology transfer, while paying insufficient attention to the underlying socio-political stressors and power dynamics that are often the primary drivers of vulnerability (Shilomboleni et al. 2024).
• Sustainable Livelihoods Approach: Resilience thinking in the African context also frequently draws upon the concepts of the sustainable livelihoods framework, which analyses how people secure their basic needs by examining their access to five core types of capital assets: human, social, natural, physical and financial (Nyamwanza 2012). Therefore, resilience assessment tools often use these five capitals as indicator domains to provide a holistic view of a household’s or community’s capacity to withstand shocks. This framework helps to structure the analysis of adaptation strategies, such as crop diversification (improving natural and financial capital) or soil and water conservation (improving natural capital) (Shilomboleni et al. 2024).

A typology of climate-resilient assessment tools and their challenges

Increasing impact in aiding smallholder farmers to build their resilience and adapt to climate change requires learning from the impact evaluation efforts of previously implemented programmes that aimed to strengthen climate resilience. This requires critically evaluating currently available climate resilience assessment tools used in the context of climate change and agricultural development with the goal of identifying patterns of best practice in methods of measurement and indicator choices. Understanding how interventions designed to strengthen climate resilience are evaluated for impact can provide lessons that can be used to inform the design and effective implementation of future programmes to improve upon this impact.

Most of the tools that have been used to assess smallholder resilience in Africa can be broadly categorised into three types of assessment approaches: index-based approaches, participatory approaches and systems-based approaches.

Organisations that work within the international development sector have tried to operationalise resilience theory in their development of several assessment tools (Table 1). Despite the inherent challenges associated with measuring resilience in practical ways because of the abstract nature of the concept, these tools have value as they help with the identification of what Darnhofer, Fairweather and Moller (2010) refer to as:

[G]eneral rules of thumb that can be used by development practitioners to guide farms, the industry sector, the national agricultural system and the interconnected parts of the international food and fibre system towards a more resilient orientation. (p. 14)

One obvious rule of thumb is linking theory to practice. Although these tools attempt to link practice to theory to a certain extent, the tools share other challenges from which lessons can be learned to inform the assessment of climate resilience among smallholder farmers in the future.

Challenge 1 – Indicator choices

Despite the fact that the chosen indicators for resilience in most of the assessment tools are consistent with theoretical concepts of resilience, some indicators that are linked to resilience theory may not lead to desirable outcomes when applied in practice. For example, the Food and Agriculture Organization of the United Nation (FAO)’s Self-Evaluation and Holistic Assessment of Climate Resilience of Farmers and Pastoralists (SHARP) tool, which aims to identify areas of weak resilience at the household level, measure their exposure and provide a baseline upon which changes could be made, is conceptualised after Cabell and Oelefse’s (2012) 13 agro-ecosystem’s resilience indicators. One of these behaviour-based indicators that is included in SHARP’s indicator selection and is supposed to be an important determining factor of a farm’s resilience is ecological self-regulation. This means that a farm should be able to maintain natural plant cover, incorporate more perennials, provide a habitat for predators and parasites and align production with ecological parameters (Cabell & Oelefse 2012). The rationale behind why this is relevant for resilience is that a greater degree of ecological self-regulation can reduce the amount of external inputs required to maintain the farm system, like nutrients, water and energy, so that production is within the limits of the system’s natural resources (Cabell & Oelefse 2012).

However, if farmers leave their farms to ecologically regulate themselves and drastically reduce the amount of external inputs required to maintain a farm, this could lead to a state of resilience that is undesirable for the farmer, whereby agricultural productivity is limited, which will then have implications for the livelihoods of farmers and our global food supply. A certain degree of ecological self-regulation is essential as it can reduce the environmental impacts of agricultural production systems, but it is not sufficient to ensure the long-term economic and social sustainability of a farm. It can be argued that a farmer’s ability to intervene and manage the regulation of a farm system’s processes to suit desired outcomes is a better indicator of farm resilience than leaving the regulation completely up to nature. This demonstrates that the selection of indicators chosen to measure resilience matters. Therefore, when developing assessment tools, it might be best not to incorporate all the indicators suggested by theoretical frameworks. It is perhaps more appropriate to select those indicators that can lead to desirable forms of resilience in the context of farm systems and the primary beneficiaries of those systems (farmers) when the tool is applied on the ground, instead of forms of resilience that are not desirable for farmers.

Challenge 2 – Social and economic sustainability vs ecological sustainability

The second challenge is that most of these tools are used in the context of development that focuses more on human well-being and less on ecosystem health. Therefore, there is a tendency to measure resilience based on indicators that associate high levels of resilience with improved livelihoods and well-being. While there is no doubt that these are important for poverty and hunger reduction outcomes, prioritising them allows us to build resilience that supports economic and social sustainability in ways that erode ecological sustainability. Sustainable development requires for there to be a balance between all three aspects of sustainability (Beder 2013; Rockström et al. 2009; Steffen et al. 2015). Research carried out by the Millennium Ecosystem Assessment has found that there is a divergent trend in the relationship between human well-being and the health of ecosystems which provide essential services that support human life on earth (Millennium Ecosystem Assessment 2005).

As human well-being has increased on average, there have also been large global declines in most ecosystem services (Raudsepp-Hearne, Peterson & Bennet 2010). Without avoiding this ecological crisis, we are not likely to achieve most of our sustainable development goals (Washington et al. 2013). In all the 15 resilience assessment tools, measurable indicators of ecosystem health and functions are not easily identifiable or are just not present. Therefore, these tools are limited in their ability to adequately monitor essential ecosystem services that are important for the resilience of farm systems. Resilience assessment tools should be able to clearly monitor ecosystem health so that we can identify weaknesses in system health earlier as well as have a better idea of how and where to restore and support biodiversity and ecosystems that support human well-being. This requires that assessment tools take on a more holistic approach by not only considering the social or economic elements of a farm system but also considering the environmental aspects as well.

Challenge 3 – Measurement time scales

The third challenge is the timescale of measurement required for most of these tools. The time needed to complete the assessment of resilience with these tools can be very long, taking several weeks, months or even a year for implementation (Douxchamps et al. 2017). To achieve the effective detection of change in indicators over time, the monitoring of selected indicators needs to occur much more frequently than these tools allow. According to Barrett and Constas (2014:144), the measurement of resilience typically includes two points, before and after a shock. However, Gunderson and Holling (2001:76) point out that different time points can be chosen based on expected change rates associated with the indicators of resilience chosen to develop the tool, allowing for resilience to not only be measured just before or after specific events that cause shock or stress. Córdoba, Trivino and Calderon (2020:2) agree stating that as farm systems are complex systems that are subject to constant change and fluctuations, it should be feasible to identify their degrees of resilience much more frequently. This implies that there is a need for assessment tools that can allow for the frequent monitoring of indicators for resilience, which is important as this can help identify crucial points where interventions are necessary much quicker than is possible now with currently available tools.

Tools that focus on a selection of a few key indicators and allow for their frequent monitoring in ways that can be repeated without long paper-based surveys can be very beneficial in providing a better picture of resilience when used alongside tools that only have the capacity to perform a ‘full’ assessment of resilience occasionally. They can make the measurement of resilience simpler and provide opportunities for the constant collection of data that can be useful when the time comes for complete assessments. The use of these methods alongside each other can also allow for longitudinal studies of farm trajectories, which can yield better results for our understanding of climate resilience over time.

Research methods and design

Developing the farm resilience assessment scorecard

The FRAS was developed as part of a Master’s dissertation project. The development process was guided by the principle that the FRAS should aim to provide a simple yet robust tool for assessing climate resilience in smallholder farm systems while addressing the challenges that have been identified in the application of previous tools.

Indicator selection and conceptual measurement framework

The FRAS is based on a conceptual measurement framework that combines a set of three indicators or factors of resilience (Table 2) that include: (1) environmental management, (2) social capital and protection and (3) reflective learning.

Indicator 1: Environmental management

The first indicator that was selected to be included in the resilience measurement framework was environmental management as one component through which the resilience of a farm system could be determined. According to Milestad and Darnhofer (2003:231), environmental management is a crucial aspect of farm resilience. A farm system that exhibits high levels of environmental management is one where a farmer has knowledge and training on farming practices like CSA that aid them in managing their farms in an environmentally sustainable manner and is able to use a configuration of these practices based on their needs and desires (Holling 2001; Milestad & Darnhofer 2003). Farm systems where farmers have knowledge of good climate-smart agricultural practices, like the ones discussed, have greater buffer and adaptive capacity. This is because farmers would know what practices to use to introduce feedback in the system that can help counter or buffer disturbances. This makes environmental management a good choice for inclusion in the conceptual measurement framework, as it is an indicator through which ecosystem health could also be gauged.

Indicator 2: Social capital and protection

Social capital and protection are understood as farmers having access to a social network or some form of savings or social protection to fall back on should they need resources, information or finances to support their recovery from a shock and supply them with information to make proactive choices (Scheffer et al. 2018; Shava et al. 2010). Social capital and protection are a widely accepted measure for coping with risks in the literature, as resource-poor smallholder farmers often rely on informal networks between families and broader networks such as community organisations to access loans and insurances that can help increase their buffer capacity and reduce their vulnerability to risk in the short term (Martin-Breen & Anderies 2011; Mausch, Orr & Miller 2017). Arinaitwe (2019:35) suggests that access to savings accounts with several banks, microfinance companies and membership in credit unions, village savings organisations and farmers’ associations are good proxies through which levels of social capital and protection could be assessed as these things can help farmers access loans, insurance, inform farmers on resources they can use to deal with environmental change and facilitate cooperation to help them with resource constraints that may arise in the event of a shock. The higher the number of sources of savings and memberships in different groups and associations, the greater the likelihood of a farmer’s resilience (Ifejika Speranza 2013). Farmers who have larger and deeper social networks, more savings or diverse forms of social protection can quickly mobilise more resources to help their farm system to persist, adapt or transform in the event of a shock compared to farmers who do not have these things (Shava et al. 2010).

It is important to acknowledge, however, that the effectiveness of the social capital and protection assets measured by this indicator is contingent upon the broader enabling environment, including the communal state of physical and institutional capital. This assessment is focused on the farmer’s direct assets and networks (the micro-system), but the actual utility of these assets is mediated by a larger system (the macro-system). For instance, a farmer’s membership in a financial institution is only a viable recovery pathway if supporting physical infrastructure, such as a reliable road network, allows them to physically access that institution, especially in the aftermath of a shock. Therefore, while this indicator captures the farmer’s potential to mobilise resources, we acknowledge that the efficiency, effectiveness and sustainability of these actions are deeply intertwined with the quality of public goods and infrastructure (e.g. roads, health centres, schools) at the village or communal level.

Indicator 3: Reflective learning

It is not enough for farmers to have access to environmental management techniques and social capital and protection. These may be essential building blocks for buffer and adaptive capacity, but they do not guarantee transformative capacity, which is a key aspect of farm resilience. Darnhofer et al. (2010:44) view resilience as more likely to emerge when farmers hone their ability to learn from past experiences with shocks and use their knowledge of environmental management practices as well as their social capital to make decisions that will create path dependencies to better anticipate changes and create a desirable future for their farm system. A farmer’s agency to do this may be the most important building block for transformation capacity. The more farmers can learn from the past and from each other and share knowledge, the more capable their farm system can be of adaptation and transformation, making it more resilient. Farmers can get better at dealing with adaptation with more experience with shocks. Learning from frequent disturbances, preferably at a small scale, can increase system resilience and adaptability in the long term because farmers can ‘naturally select’ for configurations that can sustain farm functions through periods of disturbances (Milestad & Darnhofer 2003).

The scorecard

The FRAS consists of a set of six variables, with two variables for each of the three indicators. For each variable, a scoring system was developed on a scale of 0–5, where 0 represents the lowest level of resilience and 5 the highest. The full scorecard, including questions, response options and scores, is presented in Table 3.

Application and analysis

The FRAS was applied to a dataset compiled from an endline survey of 2970 smallholder farm households conducted for the Building Resilience and Adapting to Climate Change (BRACC) project, a 5-year programme funded by the United Kingdom Foreign, Commonwealth and Development Office (FCDO) and delivered in partnership with the environmental consultancy NIRAS-LTS International, along with various implementing partners. The project’s mission was to provide targeted support to the most vulnerable districts and high-priority catchments in southern Malawi to strengthen the resilience of poor and vulnerable smallholder farm households to shocks and reduce their annual dependence on humanitarian aid (Leavy 2022). The project targeted smallholder households in four districts in Southern Malawi, which were Balaka, Chikwawa, Mangochi and Phalombe. The surveyed households in these districts were classed into two groups. One group was the treatment group, which included households that received BRACC interventions, and the other group was the control group, which included households that did not receive any BRACC interventions. The application of the FRAS involved the following steps:

• Data extraction: For each of the three indicators outlined in the conceptual measurement framework, only two variables that could be used to quantify each aspect of resilience were chosen for the scorecard out of the five that were proposed in the conceptual measurement framework. This is because the data for the other variables that were excluded were not fit for purpose or good enough to support the outcomes of the study (some variables had missing data so they would not be very useful). As a result, the scorecard has a total of six variables instead of fifteen. The relevant data for the six variables were extracted from the BRACC survey dataset.
• Scoring: Each household was assigned a score for each of the six variables based on the predefined scoring system outlined in the scorecard. In addition, the scores for two variables under each indicator were summed to create a sub-scale score that focused on each indicator (ranging from 0 to 10). This should read as follows: The scores for all six variables were also summed to create an overall farm system assessment score ranging from 0 to 30 (see Table 4 and Table 5 for scoring and interpretation guidance).
• Analysis: The scores were then analysed using statistical software (in this case R) to identify patterns and trends in resilience across the 2970 households. A ‘traffic light’ system (Table 5) was used to categorise households into three resilience levels: low (red), normal (amber) and high (green).

After the analysis was complete, the results derived from the application of the FRAS were compared to the results of another resilience assessment index that had also been developed to assess the impact of the BRACC project.

Ethical considerations

This article followed all ethical standards for research without direct contact with human or animal subjects.

Results

Descriptive analysis of scorecard data

Figure 1 displays the full farm resilience assessment mean scores for smallholder farm households in each of the four districts where the BRACC project was implemented in Malawi. We can see that most smallholder farm households in all four districts have an average full farm resilience assessment score that lies between 8 and 9 points out of a total of 30 points. Although the resilience scores are generally low in all four districts, Figure 2 shows that there is a slight difference between households that were in the treatment group and those that were in the control group. In Balaka and Phalombe districts, households that were in the treatment group had higher scores compared to those that were part of the control group. All households in Chikwawa and Mangochi were part of the treatment group, and there were no houses in those districts that were part of the control group.

FIGURE 1: Full farm resilience assessment mean scores for smallholder farm households in each district.

FIGURE 2: Full farm resilience mean scores for households in control and treatment groups in each district.

We can also observe that on average, resilience scores for smallholder farm households are still low across all three of the resilience criteria listed in the scorecard in each of the four districts. Figure 3 shows that smallholder farm households in Balaka have an average score of 2.85 for environmental management, 3.45 for social capital and protection and 2.22 for reflective learning. Figure 4 shows that in Chikwawa, households have an average score of 2.58 for environmental management, 3.6 for social capital and protection and 2.52 for reflective learning. Figure 5 illustrates that in Mangochi, households had an average score of 3.21 for environmental management, 2.62 for social capital and protection and 2.56 for reflective learning. Figure 6 shows that in Phalombe, households have an average score of 3.85 for environmental management, 2.85 for social capital and protection and 2.51 for reflective learning. From these results, we can deduce that, as scores for each of the three resilience criteria are out of 10 points, the resilience of households in each of the three criteria is low in all four districts. There are no noticeable significant differences in scores between the four districts, even when we focus on how they have scored in each of the three different criteria of resilience.

FIGURE 3: Mean resilience scores for each resilience criterion in Balaka District.

FIGURE 4: Mean resilience scores for each resilience criterion in Chikwawa District.

FIGURE 5: Mean resilience scores for each resilience criterion in Mangochi District.

FIGURE 6: Mean resilience scores for each resilience criterion in Phalombe District.

Because scores are low across all three of the resilience criteria from the scorecard in all four districts, we can then focus on the resilience scores for each of the variables that make up the scores for each criterion to understand what could be behind the low scores observed in Figure 6.

Environmental management variables

Figure 7 shows that only one-third of households in the four districts in Malawi where the BRACC project was implemented have scores that place them in the high resilience category for irrigation, as they are able to irrigate all or part of their farms throughout the planting season. The rest of the households have scores that place them in the low resilience category, and these are the households that need intervention to improve their scores in this area. Figure 8 demonstrates that only 11% of the smallholder farm households have scores that place them in the high resilience category for conservation agriculture practices, as they apply more conservation agriculture practices compared to other households. Forty-eight per cent of households fall in the normal resilience category. Forty-one per cent of households have scores that place them in the low resilience category. These are the households that apply lower than normal amounts of conservation agriculture practices or none at all, and they too are in need of intervention to increase their scores in this area.

FIGURE 7: Percentage of households in each level of resilience for irrigation.

FIGURE 8: Percentage of households in each level of resilience for conservation agriculture practice.

Social capital and protection variables

Figure 9 demonstrates that 42% of households in the four districts in Malawi where the BRACC project was implemented have loan access scores that place them in the high resilience category, and 58% of households fall in the low resilience category, as they have low scores in this area and would be in need of support to improve their scores. Figure 10 shows that only 1% of households fall in the high resilience category as they have access to multiple savings accounts. Almost a third of households do not have access to some form of savings, which makes them fall in the low resilience category in this area, suggesting they are in the category of households that require interventions to improve their resilience in this area.

FIGURE 9: Percentage of households in each level of resilience for access to loans.

FIGURE 10: Percentage of households in each level of resilience for the number of savings accounts.

Reflective learning variables

Figure 11 shows that only 9% of farming households in the four districts in Malawi where the BRACC project was implemented have shock experience scores that place them in the high resilience category as they have experienced close to four or more different environmental shocks in the past 5 years. Thirty-five per cent of households fall in the low resilience category as they experience lower than normal amounts of environmental shocks compared to the other households. Fifty-six per cent of households fall in the normal resilience category. Figure 12 shows that 80% of households fall in the low resilience category when it comes to their use of climate information to change their farming behaviour, and only 2% of households demonstrate high resilience in this area.

FIGURE 11: Percentage of households in each level of resilience for experiences with environmental shocks.

FIGURE 12: Percentage of households in each level of resilience for climate information use.

Quantitative analysis of scorecard data

Hypothesis one

Alternative hypothesis: There is a significant relationship between experience with environmental shock scores and climate information use scores.

Null hypothesis: There is no significant association between experience with environmental shock scores and climate information scores.

The relationship between experience with environmental shock and the use of climate information was examined using a Spearman correlation test as our data did not meet several of the assumptions required for a Pearson correlation test (i.e. it did not follow a normal distribution). The test was conducted at a 95% confidence interval. The test returned a p-value = 0.038 and a coefficient = 0.038. From these results, we can deduce that the relationship between the two variables is positive, weak in strength and statistically significant as the p-value is lower than 0.05. Therefore, we can reject the null hypothesis, but we cannot accept the alternative hypothesis because there is a weak effect.

Hypothesis two

Alternative hypothesis: There is a relationship between social capital and protection scores and environmental management scores.

Null hypothesis: There is no association between social capital and protection scores and environmental management scores.

This relationship was also examined using a Spearman test at a 95% confidence interval. The test returned a p-value = 2.2e-16 and a coefficient = 0.20. From these results, we can deduce that there is a positive relationship between the two variables that is weak in strength and yet statistically significant as the p-value is smaller than 0.05. This means we can reject the null hypothesis; however, we cannot accept the alternative hypothesis as the effect is weak.

Hypothesis three

Alternative hypothesis: There is a relationship between reflective learning scores and environmental management scores.

Null hypothesis: There is no association between reflective learning scores and environmental management scores.

Again, this relationship was examined using a Spearman test at a 95% interval. The test returned a p-value = 2.2e-16 and a coefficient of 0.17. This means that there is a positive relationship between the two variables that is very weak. The relationship is statistically significant, and we can reject the null hypothesis. As the effect is very weak, we cannot accept the alternative hypothesis in this case either.

Summary

In summary, the descriptive analysis of the smallholder farm households shows that full farm resilience assessment scores are low in all four districts where the BRACC project was implemented in Malawi. The sub-scale scores also demonstrate that resilience is low for all three indicators of resilience in all the four districts. The quantitative analysis shows that there is no discernible relationship between the resilience scores of the variables used to test our hypotheses. Results from the application of the FRAS showed a slight improvement in resilience among households in the treatment group compared to the control group, suggesting that the BRACC project had a modest impact. The fact that the difference in impact between the two groups of households was minimal is not surprising because although the BRACC project was set to be implemented over a period of 5 years, it was only implemented over a period of 2 years as funding was cut. The sub-scale assessment scores provided more information regarding critical areas where adjustments within the farm system or interventions were needed to improve resilience compared to the full farm resilience assessment scores. Overall, the results of the FRAS were also very similar to the results obtained from the resilience assessment index that had also been developed to assess the impact of the BRACC project.

Discussion

This section provides a critical interpretation of the scores and statistical analysis derived from the application of the FRAS to 2970 smallholder households in Malawi. The findings highlight crucial lessons for the effective assessment and enhancement of climate resilience in smallholder farming systems.

Making climate resilience tangible through simplicity

The FRAS successfully demonstrates that it is possible to translate the abstract idea of resilience into a concrete, quantifiable measure through a simple tool that is not overly burdensome in terms of data collection and analysis. This simplicity is a major benefit in resource-constrained settings like sub-Saharan Africa, enabling a wider range of stakeholders, including farmers, to engage actively in the assessment process. This shift offers an opportunity to transition from a top-down, expert-led model to a more participatory and inclusive approach in evaluation practice.

Furthermore, by concentrating on a few key indicators, the FRAS offers a responsive approach that facilitates rapid assessment and decision-making without the inconvenience of lengthy, paper-based surveys. The continuous data collection capability allows for valuable longitudinal studies of farm trajectories, yielding a deeper understanding of system resilience over time, a necessity given that resilience only emerges with time.

However, the benefit of simplification is accompanied by a critical academic challenge: the inherent trade-off between simplicity and comprehensiveness. The process of selecting a limited set of indicators necessarily breaks down resilience in a way that may not fully reflect the complex, multidimensional nature of the system. Practitioners must remain aware that the FRAS provides an accessible snapshot, and its findings should ideally be augmented by more nuanced, qualitative participatory methods to capture the full context of vulnerability and adaptive capacity.

Interpreting low resilience scores and targeting interventions

The descriptive analysis of the FRAS results provided a clear, actionable baseline for resilience across the four Malawian districts. The finding that full farm resilience mean scores were low – averaging between 8.4 and 9.76 points out of a possible 30 points (Figure 1 and Figure 2) – indicates that overall resilience across the sampled smallholder farm systems is low. This suggests that these households are predominantly in the ‘Red Zone’ or ‘Low Resilience’ category (0.00 – 1.69 points per variable), indicating they are not likely to be able to recover from an environmental shock without extensive intervention.

The sub-scale assessment scores (Figure 3 to Figure 6) proved essential in identifying critical intervention points, a key benefit of the FRAS’s design. The low scores across all three resilience criteria (environmental management, social capital and protection and reflective learning) highlight a systemic vulnerability.

Specifically, the results point to critical gaps in the system’s absorptive capacity and transformative capacity:

• Absorptive Capacity Deficiencies (Social Capital): The variables under social capital and protection were notably low. Only 1% of households had access to multiple savings accounts (high resilience). A staggering 70% were in the low resilience category for savings accounts (Figure 10). As social capital and protection (such as savings and loans) are direct proxies for the buffer capacity required to cope with short-term shocks, this finding implies that the majority of smallholders lack the basic financial mechanisms to absorb a climate shock without relying on humanitarian aid or resorting to irreversible, negative coping strategies.
• Transformative Capacity Deficiencies (Reflective Learning): The results for reflective learning showed the lowest high-resilience uptake. Eighty per cent of households demonstrated low resilience in their use of climate information to change farming behaviour (Figure 12). Reflective learning is a key building block for transformative capacity, enabling farmers to anticipate change and create a desirable future for their farm system. The overwhelming lack of climate information usage suggests a significant barrier to proactive adjustment or fundamental transformation when current livelihoods become non-viable, leaving farm systems vulnerable to ‘flipping’ into undesirable states or regimes.

Modest impact and measurement time scales

The FRAS detected only a slight improvement in resilience among households in the treatment group compared to the control group (Figure 2). This finding, while modest, aligns with the empirical reality of the programme: the BRACC project was implemented for only 2 years instead of the planned five. This duration is insufficient to induce the deeper, systemic changes required to shift a system’s resilience profile, particularly the incremental adjustments associated with adaptive capacity or the fundamental shifts of transformative capacity.

This reinforces a critical challenge in resilience assessment identified in the literature: the measurement time scale. Resilience is not a static state but a dynamic set of capacities. The frequent monitoring capabilities of the FRAS – which focus on a few key indicators (a selection of six variables out of a possible fifteen) that can be repeatedly and quickly measured – are necessary to track the expected slow rates of change associated with resilience indicators over time. The observed minimal impact after 2 years highlights the need for evaluators to adopt tools like FRAS that support longitudinal data collection to capture the nuances of resilience that only emerge after prolonged exposure to shocks and interventions.

Critical analysis of indicator correlation

The quantitative analysis, conducted using Spearman correlation tests, examined the relationships between the three resilience criteria.

Hypothesis one: Shock experience and climate information use (reflective learning)

The test for Hypothesis One found a statistically significant, but very weak positive relationship between a household’s experience with environmental shocks and its subsequent use of climate information to change behaviour (p = 0.038, coefficient of 0.038). While the statistical significance allowed for the rejection of the null hypothesis, the weak coefficient indicated a minimal practical effect. This suggests that simply experiencing shocks does not strongly compel farmers to utilise available climate information to make proactive behavioural adjustments. This disconnect highlights a significant barrier to building transformative capacity; capacity-building efforts must move beyond mere exposure to risk and focus on bridging the gap between shock experience and active, informed decision-making to create desirable path dependencies.

Hypothesis two: Social capital and protection and environmental management

The relationship between social capital and protection scores and environmental management scores also yielded a statistically significant, but weak positive relationship (p = 2.2e-16, coefficient of 0.20). This weak effect indicates that while access to financial resources and social networks (social capital) is positively associated with the adoption of beneficial, sustainable farming practices (environmental management), this link is not robust. This suggests that simply providing resources or access to loans is insufficient to enhance resilience; it must be coupled with technical training and reflective learning to ensure the translation of capital assets into sustained, environmentally sound practices that improve the system’s absorptive and adaptive capacities.

Hypothesis three: Reflective learning and environmental management

The test for the final relationship found a positive, statistically significant, but very weak relationship between reflective learning scores and environmental management scores (p = 2.2e-16, coefficient of 0.17). Similar to the other hypotheses, the effect was too weak to accept the alternative hypothesis. In theory, a higher capacity for reflective learning (the ability to learn from past experiences and anticipate change) should strongly drive the adoption of new, sustainable environmental management techniques. The very weak correlation observed suggests that the knowledge gained through learning and experience is not effectively or consistently translating into practical changes on the farm, implying a constraint in the feedback loops necessary for continuous system adaptation.

The overall pattern of weak correlations across the three hypotheses suggests that the three chosen pillars of resilience, while conceptually distinct, are not yet strongly integrated within the farming systems of the study area. This points to the need for future interventions that focus on building system-level coherence rather than isolated capacities.

Implications for evaluation practice and tool design

The FRAS’s simplicity and utility serve as a direct guide for evaluation practitioners. The central lesson is the need for a mixed-method and participatory approach. The FRAS can serve as the quantitative spine for frequent monitoring, providing a rapid, objective and comparable measure. This measure must then be supplemented by participatory engagement, allowing practitioners to understand the why behind the scores (e.g. why is climate information not being used?) and ensure the assessment is genuinely empowering, not merely extractive.

Finally, the design of future tools must remain flexible, providing opportunities for co-creation where expert knowledge is combined with local perspectives to select and adjust indicators based on specific local conditions and priorities.

Conclusion

The application and critical reflection on the FRAS offer valuable insights for improving the evaluation and enhancement of climate resilience in smallholder farm systems in sub-Saharan Africa. This article argues that a streamlined, user-centric tool like the FRAS can effectively translate the complex, multidimensional theory of farm system resilience into a tangible, quantifiable and accessible measure for field practitioners and farmers alike, and that its utility can be maximised when it is also embedded within a broader, more participatory monitoring and evaluation system.

This study on the application of the FRAS contributes practical, field-based lessons to the growing body of literature on climate resilience assessment. It demonstrates that embracing simplicity and accessibility in tool design is crucial, as it empowers a wider range of stakeholders (including the farmers themselves) to engage actively in the assessment process. Furthermore, the findings highlight the imperative of fostering participatory co-creation in assessment design. This ensures that selected indicators are contextually relevant, reflective of local knowledge and serve community priorities, thereby moving evaluations beyond purely extractive donor reporting towards genuine capacity-building and local ownership. The FRAS’ simpler structure, which enables frequent monitoring, represents a significant practical contribution, as it allows for constant collection of data for longitudinal studies of farm trajectories, which can yield better results for our understanding of climate resilience in farm systems over time.

While the FRAS provides a robust, low-burden alternative for frequent resilience monitoring, it also points to several gaps in the current literature and areas for future research.

Bridging the rigour-simplicity trade-off

The FRAS, by necessity, focuses on a limited set of indicators, creating a trade-off between simplicity and comprehensiveness. Future academic work is needed to develop methodologies that can rigorously test the validity of proxy indicators like those used in the FRAS against results from more complex, data-intensive resilience assessments (like Resilience Index Measurement and Analysis mode [RIMA] or systems-based approaches). Research should investigate the minimum viable set of indicators required to capture the essential nuances of resilience across diverse agro-ecological contexts without overburdening users.

Integrating ecological and social sustainability

The article highlighted the enduring challenge that existing assessment tools often prioritise socio-economic indicators (human well-being) over measurable indicators of ecosystem health and function. Future research must focus on designing and validating simple, farmer-led indicators that effectively monitor essential environmental services – such as soil health, biodiversity and water quality – that underpin ecological sustainability, ensuring resilience efforts do not inadvertently lead to a detrimental ‘lock-in’ effect.

Harnessing technology for participatory evaluation

The potential of technology integration (e.g. mobile apps) to transform assessment methods into dynamic, responsive and scalable processes was proposed. Empirical research is required to evaluate the effectiveness of such tech-enabled monitoring, evaluation and learning (MEL) systems in democratising data collection, enhancing the feedback loop between farmers and policymakers and ensuring data security and privacy within smallholder contexts.

In conclusion, the journey of developing and applying the FRAS has reinforced that pathways towards more effective, equitable and sustainable climate resilience assessments must embrace simplicity, adaptability, community participation and technology. By addressing the theoretical and methodological gaps outlined above, evaluation practice can be better positioned to support smallholder farmers in building more resilient and sustainable agricultural systems.

Acknowledgements

The author wishes to thank Dr. Gary Watmough for his invaluable supervision and guidance, particularly with the quantitative analysis presented in this article. Gratitude is also extended to Matthew McConnachie and Mackenzie Klema of NIRAS – LTS International for providing access to data from the Building Resilience and Adapting to Climate Change (BRACC) project and for their insightful input throughout the research process.

This article is based on a conference paper originally presented at the 11th African Evaluation Association (AfrEA) Conference, held in Kigali, Rwanda, from 18 to 22 March 2024. The conference paper, titled ‘Assessing and Building Climate Change Resilience in Smallholder Farm Systems: How Can We Do It Effectively?’, was subsequently expanded and revised for this journal publication. This republication is done with permission from the conference organisers.

The author, Bantu B. Mabaso, disclosed receipt of financial support from The Rockefeller Foundation through a grant to Genesis Analytics. The author declares that they have no other financial or personal relationships that may have inappropriately influenced them in writing this article.

Bantu B. Mabaso: Conceptualisation, Formal analysis, Investigation, Methodology, Visualisation, Writing – original draft, Writing – review & editing. The author confirms that this work is entirely their own, has reviewed the article, approved the final version for submission and publication, and takes full responsibility for the integrity of its findings.

Support for the AfrEA Conference climate strand, where some of this work was initially presented, was provided by The Rockefeller Foundation through a grant to Genesis Analytics.

The data that supports the findings of this study are not publicly available and belong to NIRAS International Consulting.

The views and opinions expressed in this article are those of the author and are the product of professional research. It does not necessarily reflect the official policy or position of any affiliated institution, funder, agency or the publisher. The author is responsible for this article’s results, findings and content.

References