The effective assessment of climate resilience requires establishing a clear and operational definition of the concept. Resilience theory structures analysis around three distinct but interrelated capacities:

Climate resilience is not a static state of being, but it is a dynamic set of these capacities, which allow a system – in this case, a farm – to continue to function in the face of a disturbance or disruption (Lutta et al. 2024). The function of importance in this case is the production of agricultural goods and services either for subsistence or to earn an income. A climate-resilient farm system is one that can avoid, absorb or adapt and transform in response to shocks and stresses, thereby maintaining its fundamental identity and functions or reorganising to a new, more viable state (Madin & Bamfo 2021). This means that in the event of a shock to the farm system, a resilient farm would be able to anticipate this disruption, absorb it, adapt to it or completely transform in such a way that the ecosystem services responsible for the provisioning of agricultural goods like food are still maintained.

In some instances, there can be trade-offs and tensions between the three capacities that can be difficult to contend with, meaning that there is always the danger of what Walker et al. (2006:13) refer to as a ‘lock-in’ effect. An example of this is the mining away of nutrients from soil as a farm tries to maintain production under stressed conditions when the farm can no longer produce within its local ecological carrying capacity, eventually making the system impoverished or resource-poor (Allison & Hobbs 2004). The system becomes resilient, but in a negative way that is often detrimental to the environment and human well-being, and this is called a ‘lock-in’ effect because a return to the old state of productivity becomes highly unlikely. This occurs because the farm system can no longer withstand or persist through climate-related shocks and ‘flips’ to an undesirable state or regime where essential ecosystem services can no longer sustain a farm’s core function of providing agricultural goods. This demonstrates that resilience is not always a positive attribute. It is in these instances where transformative capacity becomes important. This is why focusing climate action efforts on building positive resilience, where smallholder farm systems stay in desirable states or regimes in which they can still maintain productivity and support the well-being of humans, is important. Supporting smallholder farmers by preventing their farm systems from ‘flipping’ into undesirable states or regimes that fail to sustain human life in the face of climate change is pertinent to achieving crucial sustainable development goals like the elimination of poverty and hunger, especially in places in the global south like sub-Saharan Africa.

In the African context, climate resilience assessment is often framed within broader development approaches, most notably climate-smart agriculture (CSA) and the sustainable livelihoods framework:

Increasing impact in aiding smallholder farmers to build their resilience and adapt to climate change requires learning from the impact evaluation efforts of previously implemented programmes that aimed to strengthen climate resilience. This requires critically evaluating currently available climate resilience assessment tools used in the context of climate change and agricultural development with the goal of identifying patterns of best practice in methods of measurement and indicator choices. Understanding how interventions designed to strengthen climate resilience are evaluated for impact can provide lessons that can be used to inform the design and effective implementation of future programmes to improve upon this impact.

Most of the tools that have been used to assess smallholder resilience in Africa can be broadly categorised into three types of assessment approaches: index-based approaches, participatory approaches and systems-based approaches.

Organisations that work within the international development sector have tried to operationalise resilience theory in their development of several assessment tools (Table 1). Despite the inherent challenges associated with measuring resilience in practical ways because of the abstract nature of the concept, these tools have value as they help with the identification of what Darnhofer, Fairweather and Moller (2010) refer to as:

One obvious rule of thumb is linking theory to practice. Although these tools attempt to link practice to theory to a certain extent, the tools share other challenges from which lessons can be learned to inform the assessment of climate resilience among smallholder farmers in the future.

The FRAS was developed as part of a Master’s dissertation project. The development process was guided by the principle that the FRAS should aim to provide a simple yet robust tool for assessing climate resilience in smallholder farm systems while addressing the challenges that have been identified in the application of previous tools.

The FRAS is based on a conceptual measurement framework that combines a set of three indicators or factors of resilience (Table 2) that include: (1) environmental management, (2) social capital and protection and (3) reflective learning.

The FRAS consists of a set of six variables, with two variables for each of the three indicators. For each variable, a scoring system was developed on a scale of 0–5, where 0 represents the lowest level of resilience and 5 the highest. The full scorecard, including questions, response options and scores, is presented in Table 3.

The FRAS was applied to a dataset compiled from an endline survey of 2970 smallholder farm households conducted for the Building Resilience and Adapting to Climate Change (BRACC) project, a 5-year programme funded by the United Kingdom Foreign, Commonwealth and Development Office (FCDO) and delivered in partnership with the environmental consultancy NIRAS-LTS International, along with various implementing partners. The project’s mission was to provide targeted support to the most vulnerable districts and high-priority catchments in southern Malawi to strengthen the resilience of poor and vulnerable smallholder farm households to shocks and reduce their annual dependence on humanitarian aid (Leavy 2022). The project targeted smallholder households in four districts in Southern Malawi, which were Balaka, Chikwawa, Mangochi and Phalombe. The surveyed households in these districts were classed into two groups. One group was the treatment group, which included households that received BRACC interventions, and the other group was the control group, which included households that did not receive any BRACC interventions. The application of the FRAS involved the following steps:

After the analysis was complete, the results derived from the application of the FRAS were compared to the results of another resilience assessment index that had also been developed to assess the impact of the BRACC project.

This article followed all ethical standards for research without direct contact with human or animal subjects.

Figure 1 displays the full farm resilience assessment mean scores for smallholder farm households in each of the four districts where the BRACC project was implemented in Malawi. We can see that most smallholder farm households in all four districts have an average full farm resilience assessment score that lies between 8 and 9 points out of a total of 30 points. Although the resilience scores are generally low in all four districts, Figure 2 shows that there is a slight difference between households that were in the treatment group and those that were in the control group. In Balaka and Phalombe districts, households that were in the treatment group had higher scores compared to those that were part of the control group. All households in Chikwawa and Mangochi were part of the treatment group, and there were no houses in those districts that were part of the control group.

We can also observe that on average, resilience scores for smallholder farm households are still low across all three of the resilience criteria listed in the scorecard in each of the four districts. Figure 3 shows that smallholder farm households in Balaka have an average score of 2.85 for environmental management, 3.45 for social capital and protection and 2.22 for reflective learning. Figure 4 shows that in Chikwawa, households have an average score of 2.58 for environmental management, 3.6 for social capital and protection and 2.52 for reflective learning. Figure 5 illustrates that in Mangochi, households had an average score of 3.21 for environmental management, 2.62 for social capital and protection and 2.56 for reflective learning. Figure 6 shows that in Phalombe, households have an average score of 3.85 for environmental management, 2.85 for social capital and protection and 2.51 for reflective learning. From these results, we can deduce that, as scores for each of the three resilience criteria are out of 10 points, the resilience of households in each of the three criteria is low in all four districts. There are no noticeable significant differences in scores between the four districts, even when we focus on how they have scored in each of the three different criteria of resilience.

Because scores are low across all three of the resilience criteria from the scorecard in all four districts, we can then focus on the resilience scores for each of the variables that make up the scores for each criterion to understand what could be behind the low scores observed in Figure 6.

Figure 7 shows that only one-third of households in the four districts in Malawi where the BRACC project was implemented have scores that place them in the high resilience category for irrigation, as they are able to irrigate all or part of their farms throughout the planting season. The rest of the households have scores that place them in the low resilience category, and these are the households that need intervention to improve their scores in this area. Figure 8 demonstrates that only 11% of the smallholder farm households have scores that place them in the high resilience category for conservation agriculture practices, as they apply more conservation agriculture practices compared to other households. Forty-eight per cent of households fall in the normal resilience category. Forty-one per cent of households have scores that place them in the low resilience category. These are the households that apply lower than normal amounts of conservation agriculture practices or none at all, and they too are in need of intervention to increase their scores in this area.

Figure 9 demonstrates that 42% of households in the four districts in Malawi where the BRACC project was implemented have loan access scores that place them in the high resilience category, and 58% of households fall in the low resilience category, as they have low scores in this area and would be in need of support to improve their scores. Figure 10 shows that only 1% of households fall in the high resilience category as they have access to multiple savings accounts. Almost a third of households do not have access to some form of savings, which makes them fall in the low resilience category in this area, suggesting they are in the category of households that require interventions to improve their resilience in this area.

Figure 11 shows that only 9% of farming households in the four districts in Malawi where the BRACC project was implemented have shock experience scores that place them in the high resilience category as they have experienced close to four or more different environmental shocks in the past 5 years. Thirty-five per cent of households fall in the low resilience category as they experience lower than normal amounts of environmental shocks compared to the other households. Fifty-six per cent of households fall in the normal resilience category. Figure 12 shows that 80% of households fall in the low resilience category when it comes to their use of climate information to change their farming behaviour, and only 2% of households demonstrate high resilience in this area.

In summary, the descriptive analysis of the smallholder farm households shows that full farm resilience assessment scores are low in all four districts where the BRACC project was implemented in Malawi. The sub-scale scores also demonstrate that resilience is low for all three indicators of resilience in all the four districts. The quantitative analysis shows that there is no discernible relationship between the resilience scores of the variables used to test our hypotheses. Results from the application of the FRAS showed a slight improvement in resilience among households in the treatment group compared to the control group, suggesting that the BRACC project had a modest impact. The fact that the difference in impact between the two groups of households was minimal is not surprising because although the BRACC project was set to be implemented over a period of 5 years, it was only implemented over a period of 2 years as funding was cut. The sub-scale assessment scores provided more information regarding critical areas where adjustments within the farm system or interventions were needed to improve resilience compared to the full farm resilience assessment scores. Overall, the results of the FRAS were also very similar to the results obtained from the resilience assessment index that had also been developed to assess the impact of the BRACC project.

The FRAS successfully demonstrates that it is possible to translate the abstract idea of resilience into a concrete, quantifiable measure through a simple tool that is not overly burdensome in terms of data collection and analysis. This simplicity is a major benefit in resource-constrained settings like sub-Saharan Africa, enabling a wider range of stakeholders, including farmers, to engage actively in the assessment process. This shift offers an opportunity to transition from a top-down, expert-led model to a more participatory and inclusive approach in evaluation practice.

Furthermore, by concentrating on a few key indicators, the FRAS offers a responsive approach that facilitates rapid assessment and decision-making without the inconvenience of lengthy, paper-based surveys. The continuous data collection capability allows for valuable longitudinal studies of farm trajectories, yielding a deeper understanding of system resilience over time, a necessity given that resilience only emerges with time.

However, the benefit of simplification is accompanied by a critical academic challenge: the inherent trade-off between simplicity and comprehensiveness. The process of selecting a limited set of indicators necessarily breaks down resilience in a way that may not fully reflect the complex, multidimensional nature of the system. Practitioners must remain aware that the FRAS provides an accessible snapshot, and its findings should ideally be augmented by more nuanced, qualitative participatory methods to capture the full context of vulnerability and adaptive capacity.

The descriptive analysis of the FRAS results provided a clear, actionable baseline for resilience across the four Malawian districts. The finding that full farm resilience mean scores were low – averaging between 8.4 and 9.76 points out of a possible 30 points (Figure 1 and Figure 2) – indicates that overall resilience across the sampled smallholder farm systems is low. This suggests that these households are predominantly in the ‘Red Zone’ or ‘Low Resilience’ category (0.00 – 1.69 points per variable), indicating they are not likely to be able to recover from an environmental shock without extensive intervention.

The sub-scale assessment scores (Figure 3 to Figure 6) proved essential in identifying critical intervention points, a key benefit of the FRAS’s design. The low scores across all three resilience criteria (environmental management, social capital and protection and reflective learning) highlight a systemic vulnerability.

Specifically, the results point to critical gaps in the system’s absorptive capacity and transformative capacity:

The FRAS detected only a slight improvement in resilience among households in the treatment group compared to the control group (Figure 2). This finding, while modest, aligns with the empirical reality of the programme: the BRACC project was implemented for only 2 years instead of the planned five. This duration is insufficient to induce the deeper, systemic changes required to shift a system’s resilience profile, particularly the incremental adjustments associated with adaptive capacity or the fundamental shifts of transformative capacity.

This reinforces a critical challenge in resilience assessment identified in the literature: the measurement time scale. Resilience is not a static state but a dynamic set of capacities. The frequent monitoring capabilities of the FRAS – which focus on a few key indicators (a selection of six variables out of a possible fifteen) that can be repeatedly and quickly measured – are necessary to track the expected slow rates of change associated with resilience indicators over time. The observed minimal impact after 2 years highlights the need for evaluators to adopt tools like FRAS that support longitudinal data collection to capture the nuances of resilience that only emerge after prolonged exposure to shocks and interventions.

The quantitative analysis, conducted using Spearman correlation tests, examined the relationships between the three resilience criteria.

The FRAS’s simplicity and utility serve as a direct guide for evaluation practitioners. The central lesson is the need for a mixed-method and participatory approach. The FRAS can serve as the quantitative spine for frequent monitoring, providing a rapid, objective and comparable measure. This measure must then be supplemented by participatory engagement, allowing practitioners to understand the why behind the scores (e.g. why is climate information not being used?) and ensure the assessment is genuinely empowering, not merely extractive.

Finally, the design of future tools must remain flexible, providing opportunities for co-creation where expert knowledge is combined with local perspectives to select and adjust indicators based on specific local conditions and priorities.

The FRAS, by necessity, focuses on a limited set of indicators, creating a trade-off between simplicity and comprehensiveness. Future academic work is needed to develop methodologies that can rigorously test the validity of proxy indicators like those used in the FRAS against results from more complex, data-intensive resilience assessments (like Resilience Index Measurement and Analysis mode [RIMA] or systems-based approaches). Research should investigate the minimum viable set of indicators required to capture the essential nuances of resilience across diverse agro-ecological contexts without overburdening users.

The article highlighted the enduring challenge that existing assessment tools often prioritise socio-economic indicators (human well-being) over measurable indicators of ecosystem health and function. Future research must focus on designing and validating simple, farmer-led indicators that effectively monitor essential environmental services – such as soil health, biodiversity and water quality – that underpin ecological sustainability, ensuring resilience efforts do not inadvertently lead to a detrimental ‘lock-in’ effect.

The potential of technology integration (e.g. mobile apps) to transform assessment methods into dynamic, responsive and scalable processes was proposed. Empirical research is required to evaluate the effectiveness of such tech-enabled monitoring, evaluation and learning (MEL) systems in democratising data collection, enhancing the feedback loop between farmers and policymakers and ensuring data security and privacy within smallholder contexts.

In conclusion, the journey of developing and applying the FRAS has reinforced that pathways towards more effective, equitable and sustainable climate resilience assessments must embrace simplicity, adaptability, community participation and technology. By addressing the theoretical and methodological gaps outlined above, evaluation practice can be better positioned to support smallholder farmers in building more resilient and sustainable agricultural systems.