Article Information

Author:
Hailemichael Taye1

Affiliation:
1International Livestock Research Institute (ILRI), Addis Ababa, Ethiopia

Correspondence to:
Hailemichael Taye

Postal address:
5689, Addis Ababa, Ethiopia.

Dates:
Received: 27 Mar. 2013
Accepted: 19 Aug. 2013
Published: 30 Sept. 2013

How to cite this article: Taye, H., 2013, ‘Evaluating the impact of agricultural extension programmes in sub-Saharan Africa: Challenges and prospects’, African Evaluation Journal 1(1), Art. #19, 9 pages. http://dx.doi.org/10.4102/
aej.v1i1.19

Copyright Notice:
© 2013. The Authors. Licensee: AOSIS OpenJournals.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Evaluating the impact of agricultural extension programmes in sub-Saharan Africa: Challenges and prospects
In This Original Research...
Open Access
Abstract
Abstrait
Introduction
Research method and design
Results
   • Agricultural production and productivity in sub-Saharan Africa (SSA)
   • Agricultural extension impact evaluation results
Discussion
   • Challenges in agricultural extension impact evaluations
      • The nature of impact evaluation
      • The nature of extension
      • The nature of agriculture in SSA
Methodological factors
   • Indicators used to measure outcomes
   • Impact evaluation designs
   • Sampling
   • Statistical analysis
   • Evaluation research design and implementation
   • Baseline and reliable data
Capacity-related factors
   • Shortage of skilled manpower to conduct rigorous impact evaluation
   • Poor monitoring and evaluation systems
   • Budget constraints for impact evaluation studies
Future prospects of impact evaluation of agricultural extension programmes
Conclusions
Acknowledgements
   • Competing interests
References
Appendix 1
Abstract

Background: In sub-Saharan Africa (SSA), impact evaluation has been used to assess whether agricultural extension interventions have brought the intended result or to establish causal linkages between interventions and outcomes. However, there is some scepticism about the validity and reliability of the results of the impact evaluation reports due to some contradictory and exaggerated results.

Objectives: This article analyses some impact evaluation studies conducted in SSA as to why contradictions and exaggerations are manifested in some reports and what would be the future prospects of impact evaluation of agricultural extension programmes in the region.

Methods: Impact evaluation reports and results of agricultural extension programmes from 10 SSA countries were reviewed and analysed based on impact evaluation principles and theories.

Results: The results show that most of the evaluations reported positive impacts. There are also conflicting reports on extension performance. The fact that the overwhelming majority of impact evaluation reports claim positive extension impacts is not in line with the reports on agricultural productivity growth in the region. There are various reasons for overestimated impacts and contradictory results, which include use of poor impact evaluation methodologies, lack of reliable data and insufficient capacity to conduct rigorous impact evaluations.

Conclusion: Due to these challenges and the shift in agricultural research and extension approaches, it is recommended that rather than investing effort in trying to prove impact, greater attention should be given to improving impact as well as using other innovative monitoring and evaluation (M&E) and learning tools that consider the dynamic nature of agricultural development.

Abstrait

Évaluer l’impact des programmes de développement agricole en Afrique subsaharienne: défis et perspectives

Présentation: En Afrique subsaharienne (ASS), l’évaluation de l’impact a été utilisée afin de déterminer si les interventions relatives au développement agricole avaient produit les résultats escomptés ou afin de déterminer les liens de cause à effet entre les interventions et les résultats obtenus. On observe cependant un certain scepticisme quant à la validité et à la fiabilité des résultats des rapports sur l’évaluation de l’impact en raison de leur caractère contradictoire et exagéré.

Objectifs: Cet article analyse des études d’évaluation de l’impact en ASS afin de déterminer pourquoi les contradictions et les exagérations sont manifestes dans certains rapports, et quelles seraient les perspectives futures des programmes d’évaluation de l’impact du développement agricole dans la région.

Méthodes: Les rapports et résultats en matière d’évaluation de l’impact des programmes d’extension agricole tirés de dix pays d’ASS ont été examinés et analysés à partir de principes et théories de l’évaluation de l’impact.

Résultats: La plupart des évaluations ont révélé des impacts positifs. On note aussi des rapports contradictoires sur la performance du développement. Le fait que la grande majorité des rapports d’évaluation de l’impact fasse état d’impacts positifs sur le développement n’est pas conforme aux rapports sur la croissance de la productivité agricole dans la région. Il existe plusieurs raisons à la surestimation des impacts et aux résultats contradictoires, et notamment le recours à des méthodologies d’évaluation de l’impact de mauvaise qualité, l’absence de données fiables et une capacité insuffisante pour procéder à des évaluations rigoureuses de l’impact.

Conclusion: En raison de ces défis et du changement observé dans les approches à la recherche et au développement agricole, il est recommandé qu’une plus grande attention soit accordée à l’amélioration de l’impact et à l’utilisation d’autres outils de suivi et d’évaluation (S&E) et d’apprentissage innovants tenant compte de la nature dynamique du développement agricole, plutôt que d’essayer de prouver l’impact.

Introduction

Agricultural extension has long been seen as a key element for enabling farmers to obtain information and technologies that can improve their livelihoods (Purcell & Anderson 1997) and is recognised as an important factor in promoting agricultural development (Birkhaeuser et al. 1991; Anderson & Feder 2007). Hence, from a development policy perspective, investments in extension services are considered as potentially important tools for improving agricultural productivity and increasing farmers’ incomes (Anderson 2007). Consequently, sub-Saharan African (SSA) countries have been implementing various agricultural extension programmes to improve agricultural production and productivity, farm income and rural livelihoods.

Agricultural extension theories have shown changes over time, based on changes in agricultural development paradigms. As a result, a variety of extension approaches have been promoted in the past several decades. Early models that focused on transfer of technology characterised by ‘top-down’, ‘linear’, ‘rigid’ approaches were criticised for their reductionist perspectives and the passive role of farmers. These approaches also failed to recognise the diversity of the socio-economic and institutional environments and other factors affecting farmers (Chambers & Ghildyal 1984; Birner et al. 2006). Since the 1970s, however, there was a change of extension models due to change in agricultural development discourses over the years. Hence, extension models such as training and visit (T&V), participatory approaches, farmer field schools (FFSs), ICT-based on-line advisory services to farmers and promotion of model farms (Waddington et al. 2010) have been used in different countries.

Impact evaluations were conducted to test the success and outcomes of various extension interventions and approaches in sub-Saharan African countries (SSA). The difficulty of tracing the causal relationship between extension input and its impact is appreciated by various authors (Anderson 2007; Anandajayasekeram et al. 2008; Davis 2008). This article, based on analysis of a number of impact evaluation studies conducted in SSA, attempts to identify challenges and future prospects associated with the conduct of impact evaluations of agricultural extension programmes in the region.

Research method and design

This article is based on a review of impact evaluation reports on agricultural extension programmes in SSA. The results of impact evaluations conducted in 10 SSA countries including Kenya, Somali, Ivory Coast, Rwanda, Burkina Faso, Uganda, Zimbabwe, Mozambique, Ethiopia, and Tanzania were collected from various websites for the analysis. In addition, agricultural productivity trends reported by various authors on a variety of reports and websites are included. This study is mainly based on qualitative analysis. The quantitative data are analysed using simple descriptive statistics.

Results

Agricultural production and productivity in sub-Saharan Africa (SSA)
The performance of the agricultural sector in SSA has been unsatisfactory and lagging behind its population growth. For instance, the annual growth rate of agriculture value-added in SSA was only 1.9% during 1965–1980 and fell to 1.7% during 1980–1992 whilst the population growth rate increased from 2.7% to 3% during 1965–1980 and 1980–1992 respectively. Likewise, the per capita cereal production in SSA consistently declined from 169 kg in 1970, to 140 kg in 1993 (Kahan & Kahan 1995). In general, the per capita agricultural production has been declining in SSA in contrast with increasing trends in Latin America and East Asia (Haggblade et al. 2004; Haggblade & Hazell 2010). Due to the poor performance of the agriculture sector in the region, food insecurity has increased significantly. For example, the number of chronically under-nourished people increased from 168 million in 1990–1992 to 194 million in 1997–1999 (NEPAD 2003). Consequently, food aid in the region oscillated between 2.6 million tons in 1996 and 5.2 million tons in 2003 (WFP 2005).

Yields per ha have been stagnant in the SSA region, particularly for cereals, in contrast to substantial yield increases in other regions of the world from 1960 to 2005 (Staatz & Dembele 2007). This means that even if there is yield production growth, it is mainly due to area expansion, awhich shows limited technological change in the agriculture sector.

Agricultural extension impact evaluation results
Despite the fact that agricultural productivity in SSA is lagging behind the rest of the world, the majority of impact evaluations conducted in the region have reported a positive impact of agricultural extension programmes. Of the 21 impact evaluation results analysed, 15 (71%) impact evaluation studies reported positive impact (Appendix 1). Similar findings are reported in other reviews. For instance, of 48 studies of extension reviewed by Birkhaeuser et al. (1991), 36 (75%) showed significantly positive results. In addition, high rates of return to extension in the range of 13% – 500% were reported in various impact evaluation studies, including Birkhaeuser et al. (1991).

The impact evaluations in SSA have asserted that the effectiveness of various agricultural extension models is context specific (Appendix 1). For instance, Purcell and Anderson (1997) evaluated T&V extension models in five SSA countries and found that the model had a satisfactory effect in three of the countries (Kenya, Somali and Zimbabwe) whereas it had an unsatisfactory effect in Ivory Coast and Rwanda. The Uganda National Advisory Services (NAADS) was reported to have positive impact in terms of availability and quality of services but there was no significant difference in terms of yield growth (Benin et al. 2007). Similarly, Davis et al. (2010) found that FFS extension models had mixed results in the East African region. The model brought significant change in terms of productivity and income in Kenya and Tanzania whilst it had non-significant impact on the same variables in Uganda.

Apart from the result of the impact evaluations, which is inconsistent with the agricultural growth trend in the region, some of the impact evaluation studies are also contradictory. Bindlish and Evenson (1997) in their study of the T&V programme reported a high and significant rate of returns to extension in Kenya. However, Gautam and Anderson (1999) rejected this result, reasoning that the evaluation suffered from data and specification problems. Using the same data, they showed that the finding was invalid and the rate of return was not significant. Similarly, whilst Ayele et al. (2005) and IFPRI (2008) reported a positive and significant effect of extension in Ethiopia, another study conducted by EEA/EEPRI (2006) asserted that the Ethiopian extension programme suffers from various problems and its impact on farm income, productivity, food security and poverty is not significant.

In Mozambique, Walter et al. (2004) reported that agricultural extension had a negative and statistically insignificant impact on individual and household income. Neither did agricultural extension services lead to any perception of improved economic condition in the household. They stipulated that the absence of positive and statistically significant results on household and community variables could be due to limited access to improved inputs and location-specific adapted technologies. To the contrary, ECON Analysis (2005) rejected this finding and reported that extension services in Mozambique had positive effects on rural livelihoods through promotion of new varieties, inputs such as natural pesticides and soil conservation practices.

Due to exaggerations and contradictions in similar reports, some researchers are sceptical of the results of evaluation studies conducted in sub-Saharan Africa (Davis 2008). There are different reasons for such contradictions and exaggerations in impact evaluation studies conducted on agricultural extension programmes in the region. These are related to challenges and limitations that emanate from the nature of impact evaluation, extension science, agriculture, impact evaluation methodologies and capacity that will be discussed in the next section.

Discussion

Challenges in agricultural extension impact evaluations
There are various challenges that affect the conduct and results of impact evaluation of agricultural extension programmes.

The nature of impact evaluation
There are several definitions of impact evaluation. For the purpose of this article, impact evaluation is defined as an attempt to create a causal linkage (attribute) for given extension outcomes to extension interventions/inputs using quantitative methods. Many suggest that it is difficult to attribute a change in outcomes to a specific intervention, since there are so many different factors involved (White 2009). Hence, controlling the extraneous and design effects for estimating the net impact which could be attributed to the interventions is one of the challenges for impact evaluators. The other inherent challenge in impact evaluation is estimating the counterfactual. That is, what would have happened to those receiving the intervention if they had not received the programme. This requires control groups which need to be as similar as possible to the treatment group, except for the intervention.

The nature of extension
Despite differences in modalities and success across countries, extension refers to the conscious use of communication of information to help people form sound opinions and make good decisions (Van den Ban & Hawkins 1996). The adoption of technologies and better practices is the core of extension interventions. Target farmers in extension need to pass in a certain process before achieving their ultimate goal of improving productivity. The process includes awareness, knowledge, adoption of technology or practice and change in farmers’ productivity (FAO 1997). To this end, various extension methods are used, such as extension contacts, meetings, mass media, demonstrations, etc. In addition to these, there are informal networks where technologies, knowledge and practices are spread to other farmers.

Bringing agricultural growth has been a main agenda for most sub-Saharan African countries. Investment in extension services is an important tool for improving agricultural productivity and increasing farmers’ incomes (Anderson 2007). Hence, bilateral and multilateral donors have been supporting efforts of developing countries in providing extension services. Therefore, most SSA countries have been providing agricultural extension services for the last several decades even if the modalities have shown changes over the years.

Small-scale extension initiatives in Africa have been supported by the World Bank since 1970, when it started financing integrated rural development projects. Following the shift in its emphasis from integrated rural development projects to the development of national institutions in the early 1980s, the Bank began to support reorganisation of national agricultural extension systems in Africa.

The T&V model was first adopted by several dozen countries in Africa in the eighties. In practice, the T&V system was effective in disseminating Green Revolution technology, especially in the high-potential, irrigated areas, in promoting very specific packages. T&V has been criticised for high recurrent cost and financially unsustainable (Anderson, Feder & Ganguly 2006). The extension debate shifted in the late eighties and nineties to a new model: the farmer field school (FFS) (Anderson et al. 2006). Farmer field schools were introduced into sub-Saharan African in the mid-1990s. Although many positive reports exist on the benefits of the FFS approach, some studies have called into question their overall impact and financial sustainability (Davis 2008).

Most extension models have common characteristics such as addressing farmers without restrictions, using mass communication methods, extension agent-farmers’ contacts, farmers-to-farmer extension, etc. These typical characteristics affect impact evaluation in the following ways:

Lack of comparable control group: Agricultural extension programmes mostly are run nationwide and it is difficult to establish control groups (or exclude some from the benefits in order to establish a control group) from operational and ethical considerations. Furthermore, there is always spill-over of technology, knowledge and information from programme beneficiaries to non-beneficiaries, making it hard to find an ‘uncontaminated’ counterfactual. Sometimes evaluators use non-participants as a control group. This may lead evaluators to two types of erroneous conclusions.

a. There is non-significant impact: This is referred as a false negative or type II error in the specialised literature. As information, knowledge and technology diffuse from participants to non-participants, the difference in outcome between the two groups could be similar. This could lead to an underestimation of the effective impact.

b. There is significant impact: This is referred as false positive or type I error in the specialised literature. When selecting control groups for impact evaluation, they are expected to be comparable to the treatment group. This means that both groups should be similar (on other variables), except in their access to the programme intervention and, as a result, characterised by a different use of the information, knowledge and technologies of the extension programme. Extension programmes don’t usually have restrictions and are available to every farmer. Hence, if the technology is good, most farmers will adopt it, except those farmers known as ‘laggards’. These segments of the population are not expected to perform well, even in the absence of extension programmes. That means there are other individual factors that differentiate the beneficiary and non-beneficiary groups other than the extension programme, which exaggerates the net impact.

Selection bias: When the differences in outcomes between the treatment and the control group are due to pre-existing differences between the treatment and control groups (in which case randomization did not work), the evaluation conclusions are affected by a validity threat that is normally referred to as selection bias. Peoples involvement in agricultural extension programmes is based on interest and resource availability. Additionally, the ability to try new technology requires the capacity to take risks. The better-offs and the educated are those with better information access and risk-taking capacity. These people are likely to be successful even without the extension programme intervention. This will overestimate the impact of the intervention. There is also de-selection bias which is the opposite of the above discussion.

Endogeneity in extension-farmer interaction: Extension contact, which plays a central role in the success of extension programmes, is affected by farmers’ characteristics and actions. For instance, more productive farmers tend to seek more extension contacts and get more information and technologies than others. Similarly, extension agents also may seek contacts with model farmers who are good performers even in the absence of extension contacts. In this case, estimating the impact of extension programmes could be biased upward, which is attributed to the superior characteristics and actions of farmers.

Other sources of information and technology: There are other sources of information and technology, apart from the extension programme, available to farmers. Farmers use various technologies from the market and/or information they get from other media to improve their productivity. It is not easy to differentiate between the impacts of technologies disseminated by the extension programme and from other sources. This is particularly difficult in SSA, where most farmers are uneducated and cannot easily identify the technology provider. Hence, the impact of the extension intervention under evaluation may be exaggerated.

Extension to be effective needs other support services: the success of extension programmes with the ultimate goal of improving productivity depends on relevant and appropriate technology (research), timely provision of inputs, good market outlets for agricultural produces, good transportation facilities, good credit services, etc. Success/failure of these services is related to the success/failure of extension programmes. Unless their effects are not controlled, it may lead to under or overestimation of the effects of investment in extension services.

The nature of agriculture in SSA
Agriculture, especially in sub-Saharan African countries is highly dependent on nature and it is typically complex, diverse and risk prone. Agricultural production and productivity depends on various natural factors such as season, agro-ecology, climate, soil type, water availability, rainfall pattern, pests and diseases, etc., which vary over time and space. The impact of agricultural extension programmes could be over or under-estimated if these effects are not controlled.

Methodological factors

Indicators used to measure outcomes
This is related to the definition of impact evaluation. There are two debating definitions given for impact evaluation. The first definition of ‘impact’ refers to the final level of the causal chain. On the other hand, others working in impact evaluation define impact as the difference in the indicator of interest (Y) with the intervention (Y1) and without the intervention (Y0). That is, impact = Y1 − Y0. In this sense, impact evaluation is a study which tackles the issue of attribution by identifying the counterfactual value of Y (Y0) in a rigorous manner. The two definitions could overlap if Y is an outcome indicator (White 2009).

In agricultural extension programmes, the knowledge and information acquired is an output whilst adoption of technologies is an outcome and the final impact is change in productivity. Most impact evaluation studies focus on one or two of this hierarchy of objectives but not the whole process (FAO 1997). However, according to White (2009) the job of a quality impact evaluation is to trace the causal chain from inputs to outcomes and impacts. Hence, the final outcome in extension programmes should be measured in terms of productivity. Measuring extension impact by using change in knowledge and awareness (e.g. Oleru et al. 2005) would overestimate the impact of extension programmes. Sometimes, other indicators that are not related to extension are used to evaluate agricultural extension impact. For example, in Ethiopia extension effectiveness was measured using input distributed (Spielman 2008).

Impact evaluation designs
Experimental and quasi-experimental designs are termed as rigorous, even if the former are more rigorous. However, impact evaluations in SSA have shortfalls in employing these evaluation designs, which leads to under or over-estimation of real impacts. Most of the impact evaluations have used regression models to show and compare relationships between extension interventions and outcomes (e.g. Temu et al. 2005; Walker et al. 2004; ECON Analysis 2005; Romani 2003; Owens, Hoddinott & Kinsey 2003), whilst others used longitudinal impact evaluation (difference in difference approach) with quasi-experimental methods (e.g. Davis et al. 2010 ), and some used simple with and without designs (e.g. EEA/EEPRI 2006; Benin et al. 2007; Oleru et al. 2005).

Sampling
The sample should be random and large enough so that it represents the population and provides adequate statistical power. However, impact evaluation studies in sub-Saharan Africa either concentrate in successful areas or the samples are not representative, are too small in size and don’t have enough statistical power to give valid conclusions. For example, IFPRI (2008) reported a positive impact of extension and roads on poverty and consumption growth in Ethiopia based on the analysis of only 15 villages. Likewise, Econ Analysis (2005) used a sample of 480 households to come to the conclusion that agricultural extension services have positive effects on rural livelihoods in Mozambique and have tried to challenge the findings of Walker et al. (2004) that reported agricultural extension had non-significant impact on rural income in that country. Besides, due to purposive inclusion of villages with extension coverage in the sampling, there is high probability of a skewed sample and the sample could have better experience with extension than the true national coverage.

Statistical analysis
Regression techniques are the preferred method for analysis in impact evaluations so that the effects of confounding factors could be controlled. But in some impact evaluations (e.g. EEA/EEPRI 2006) carried out in sub-Saharan African countries, simple mean and averages are used to estimate impacts, which will over or under-estimate the impact of extension programmes. Others used OLS estimates, means and chi square and t-tests for analysing the data (e.g. Walker et al. 2004, Oleru et al. 2005, Benin et al. 2007).

Evaluation research design and implementation
Poor design and implementation of the evaluation process affects the results of impact evaluations. Design effects that include factors observed in the research process (such as errors of measurement, sampling variations and inconsistency in data collection) could under or over-estimate results. In SSA, evaluations are carried out in a hurry using a tight schedule because of lack of skilled manpower, budget, time and logistics, and this can pose a threat to the validity of the evaluation results.

Baseline and reliable data
In SSA it is reported that there is lack of base-line data for development projects in general and extension in particular. For example, impact evaluations conducted by EEA/EEPRI (2006) did not use base-line surveys because of absence of the same. Others tried to reconstruct base-line values in final evaluations (e.g. Davis et al. 2010; Benin et al. 2007). Lack of base-line data greatly affects the rigorousness of the impact evaluation. Besides, getting reliable data is a huge challenge in the region. Gautam and Anderson (1999) have challenged that there are problems with the data used by Bindlish and Evenson (1997) to evaluate Kenyan T&V extension.

Capacity-related factors

There are various capacity-related issues in SSA related to M&E in general and impact evaluation in particular.

Shortage of skilled manpower to conduct rigorous impact evaluation
Conducting rigorous impact evaluation requires skilled manpower in the area of impact evaluation methodologies and sophisticated statistical methods. However, in the region there is severe shortage of qualified manpower in the area of M&E in general and impact evaluation in particular.

Poor monitoring and evaluation systems
Impact data need to be continuously collected and available for use. However, in these countries there is poor capacity to collect and store quality data. The M&E systems in the region are reported to be poorly organised. For instance, in the Ethiopian extension programme (PADETES) the M&E system is poor (EEA/EEPRI 2006) and the information system is non-existent for the Kenyan national extension programme (NEP) (Bindlish & Evenson 1997).

Budget constraints for impact evaluation studies
Impact evaluation requires a large budget. However, as most SSA countries are poor, impact evaluation studies are under-funded. Due to this, impact evaluators rush to finish the study using small samples, within short periods of time and using less competent professionals.

These challenges and factors affect the validity and reliability of agricultural extension impact evaluation results in SSA. Hence, due to these challenges and the contemporary shift in agricultural extension paradigms, there is a need to re-examine the practice of impact evaluation on agricultural extension programmes.

Future prospects of impact evaluation of agricultural extension programmes

Conventionally, agricultural extension has been considered as a ‘linear model’ of innovation, where agricultural innovations are developed by research centres which are assumed to be transferred to farmers by public agricultural extension organisations. In this model, farmers are passive recipients of technologies. Recently, however, it is understood that agricultural research and extension is not a linear process, but a dynamic and complex process with institutional pluralism. Hence, current trends and changes in agricultural research and extension processes require more dynamic and systemic research and extension approaches characterised by recognition and utilisation of multiple sources of knowledge and viewing extension as a co-learning process.

This paradigm shift emanates from challenges faced by agricultural extension organisations, farmers and agriculture due to the dynamic nature of social and natural environments (Leeuwis 2004). These are related to funding, new emerging roles, development in extension theories and practices and the emergence and expansion of new communication technologies. These changes have necessitated the need for dealing with collective issues, co-designing rather than disseminating innovations, matching the technical and social dimension of innovation, catering for diverse farming and livelihood strategies, managing complexity, conflict and unpredictability, becoming learning organisations, being brokers in an era of participation, and coping with dwindling resources (Leeuwis 2004). The new agricultural extension principles include a shift from ‘transfer of technology’ to facilitation and from considering farmers as ‘beneficiaries’ to clients and stakeholders. Moreover, it recognises the importance of market demand, the private sector, pluralism and decentralisation, along with the necessary coordination and dialogue among actors (Neuchâtel Group 1999).

Such a paradigm shift in agricultural research and extension calls for a review of M&E approaches including impact evaluation. For instance, one of the developments in extension science that affects extension impact evaluation is the need to manage complexity, conflict and unpredictability. Technical and agro-ecological processes do not always behave in expected and linear ways, and at the point where the social and technical meet, many unforeseen developments may take place (Holling 1995). Furthermore, innovations are inherently messy, chaotic, complex and unpredictable (Leeuwis 2004). Hence, extension organisations need to organise their interventions in a much more adaptive and flexible way so that learning experiences and emergent developments can be incorporated in ongoing activities. This requires new forms of monitoring and evaluation including impact evaluation.

Creating causal relationships between interventions and outcomes will not be easy due to the complex, dynamic and institutional pluralism nature of the agricultural development and the new research and extension paradigm. Hence, M&E approaches which are participatory, qualitative and focused on learning would be useful. Likewise, due to the challenges related to impact evaluation mentioned above, there is a need to move from ‘proving impact’ to ‘improving impact’. The former aims at providing conclusive evidence for donors or policy-makers and the approach taken is likely to be externally led and biased towards collecting quantitative data for accountability purposes. The latter looks at how to improve practice and develop internal learning systems, with a focus on qualitative data and use of participatory tools. In line with the new development discourse, various M&E tools have been developed including outcome mapping (OM), most significant change (MSC), etc. These tools are more learning oriented, focus on contribution than attribution, participatory and flexible in nature

Conclusions

Various extension programmes have been launched in sub-Saharan Africa with the aim of improving agricultural production and productivity. However, agricultural production and productivity is not growing as it should be in the region. Despite this fact, most of the impact evaluation studies conducted on agricultural extension programmes in the region have reported positive and significant impacts. This is not only contradicts the reality in terms of agricultural productivity growth but also the impact evaluation studies reported contradictory results in a given country. The reasons for such contradictions emanate from the nature of impact evaluation, agriculture, extension, and factors related to the methodologies used for impact evaluation, data and capacity.

Attribution and estimating the counterfactual are the challenges that emanate from the nature of impact evaluation itself. The challenges related to the nature of agricultural extension programmes are: spill over effects, selection biases, endogeneity in farmers’ extension interactions, differentiating the effect of extension programme from other sources of technology or information and other complementary services such as research and the difficulty of establishing a comparable control group due to ethical and political reasons.

The complex, diverse and risk-prone nature of agriculture in sub-Saharan Africa makes it difficult to disentangle the true impact of agricultural extension services. The definition and indicators used to measure impact, application of appropriate impact evaluation design, sample size, statistical techniques used for analysis, availability of baseline data and adequate data and capacity of staff in rigours impact evaluations and availability of other resources, are other factors that affect the validity and quality of impact evaluation results.

Agricultural extension has undergone a paradigm shift, from transfer of technology (ToT) approaches to innovation systems approaches, which perceive the agricultural development process as complex and dynamic and with institutional pluralism against the conventional linear and unidirectional approaches. Hence, this new paradigm requires a shift in M&E methodologies. In line with this, impact evaluation objectives also have shown a shift from proving impact to improving impact.

Building evaluation capacity, establishing appropriate M&E systems, using impact evaluation approaches in line with the reality in SSA and a change in agricultural extensions paradigms, using easy, less costly and more flexible tools would improve shifting the objective of impact evaluation from proving to improving impact, would improve the quality, validity and usefulness of impact evaluation results in the region. For instance, outcome mapping (OM) has been reported to be a useful monitoring, evaluation and learning tool for interventions that require dynamic and complex processes and the contribution of multiple stakeholders at various levels. The tool enhances the possibility of development impact through continuously improving implementation based on feedback information and learning (Earl, Carden & Smutylo 2001).

Acknowledgements

I have received generous support from various institutions whilst writing this article. The earlier version of this article was presented at a NONIE meeting in Rome and a CTA extension conference in Nairobi. I would like to thank not only NONIE and CTA for covering the costs related to my participation in the events but also the participants of the meeting and conference for their valuable inputs that helped me to improve the article. I sincerely acknowledge all external reviewers that were involved in reviewing this manuscript. Last but not least, I would like to thank 3ie for its bursaries that gave me the chance to participate in impact evaluation training and the conference, which has been an impetus to write this article.

Competing interests
The author declares that he has no financial or personal relationship(s) which may have inappropriately influenced him in writing this article.

References

Anandajayasekeram P., Puskur R., Workneh, S. & Hoekstra D., 2008, Concepts and practices in agricultural extension in developing countries: A source book, International Food Policy Research Institute (IFPRI), Washington, DC. and International Livestock Research Institute (ILRI), Nairobi.

Anderson, J.R., 2007, ‘Agricultural advisory services.’ Background Paper for World Development Report 2008, Agriculture for Development, World Bank, Washington, DC.

Anderson, J. R. & Feder, G., 2007, ‘Agricultural extension’, in R.E. Evenson & P. Pingali (eds.), Handbook of agricultural economics, vol. 3, Agricultural development: Farmers, farm production, and farm markets, p. 2343–2378, Elsevier, Amsterdam.

Anderson, J.R., Feder, G. & Ganguly, S., 2006, ‘The rise and fall of training and visit extension: An Asian mini-drama with an African epilogue?’, in A.W. Van den Ban & R.K. Samanta (eds.), Changing roles of agricultural extension in Asian nations, p. 149–172, B.R. Publishing Company, New Delhi.

Ayele, G., Alemu, D. & Kelemework, F., 2005, ‘The provisions of rural services in Ethiopia: Characterization, impacts, and farmers’ priorities’, unpublished manuscript, International Food Policy Research Institute (IFPRI), Washington, DC.

Benin, S., Nkonya, E., Okecho, G., Pender, J., Nahdy, S., Mugarura, S., Kato, E. & Kayobyo, G., 2007, ‘Assessing the impact of the national agricultural advisory services (NAADS) in the Uganda rural livelihoods’, IFPRI Discussion Paper 00724. International Food Policy Research Institute (IFPRI), Washington, DC, viewed 27 August 2013, from http://www.ifpri.org/sites/default/files/publications/ifpridp00724.pdf

Bindlish, V. & Evenson, R.E. 1997. ‘The impact of T&V extension in Africa: The experience of Kenya and Burkina Faso’, The World Bank Research Observer 12 (2), 183–201.

Birkhaeuser, D., Evenson, R.E. & Feder, G., 1991, ‘The economic impact of agricultural extension: A review.’ Economic Development and Cultural Change 39(3), 607–640.

Birner, R., Davis, K., Pender, J., Nkonya, E., Anandajayasekeram, P., Ekboir, J., Mbabu, A., Spielman, D., Horna, D., Benin, S. & Cohen, M. 2006, ‘From “best practice” to “best fit”: A framework for analyzing pluralistic agricultural advisory services worldwide’, DSGD Discussion Paper No. 37, IFPRI, Washington, DC.

Chambers, R. & B.P. Ghildyal, 1984, ‘Agricultural research for resource-poor farmers: The “farmer first and last” model’, Paper for the National Agricultural Research Project Workshop, Hyderabad, India.

Davis, K. 2008. ‘Extension in sub-Saharan Africa: Overview and assessment of past and current models and future prospects’, Journal of International Agricultural and Extension Education 15(3), 15–28.

Davis, K., Nkonya, E., Kato, E., Mekonnen, M., Odendo, M., Miiro, R. & Nkuba, J. 2010, ‘Impact of farmer field schools on agricultural productivity and poverty in East Africa’, IFPRI Discussion Paper 00992, International Food Policy Research Institute, Washington, DC, viewed 27 August 2013, from http://www.ifpri.org/sites/default/files/publications/ifpridp00992.pdf

Earl, S., Carden, F. & Smutylo, T. 2001, Outcome mapping – Building learning and reflection into development programs, International Development Research Centre (IDRC), Ottawa.

ECON Analysis, 2005), ‘Impacts of extension services in rural Mozambique’, Report 2005-015, commissioned by World Bank, Project no. 42860 Public, viewed 27 August 2013, from http://www.econ.no/stream_file.asp?iEntityId=2001

EEA/EEPRI. 2006. Evaluation of the Ethiopian agricultural extension with particular emphasis on the participatory demonstration and training extension system (PADETES), Ethiopian Economic Association/Ethiopian Economic Policy Research Institute, Addis Ababa.

FAO, 1997, Improving agricultural extension: A reference manual, FAO, Rome.

Gautam, M. & Anderson, J.R., 1999, ‘Reconsidering the evidence on returns to T&V extension in Kenya’, Policy Research Working Paper (WPS2098), World Bank, Washington, DC, viewed 27 August 2013, from http://www-wds.worldbank.org/external/default/WDSContentServer/IW3P/IB/1999/06/04/000094946_99042105315210/additional/113513322_20041117141033.pdf

Haggblade, S. & Hazell, P.B.R., 2010, ‘Successes in African agriculture.’ IFPRI Issue Brief No. 63, International Food Policy Research Institute, Washington, DC, viewed 27 August 2013, from http://www.ifpri.org/sites/default/files/publications/ib63.pdf

Haggblade, S., Hazell, P., Kirsten, I & Mkandawire, R., 2004, ‘Building on successes in African agriculture: African agriculture – Past performance, future imperatives’, 2020 Vision Focus 12 Brief No. 1, IFPRI, Washington, DC.

Holling, C.S., 1995, ‘What barriers? What bridges?’, in L.H. Gunderson, C.S. Holling & S.S. Light (eds.), Barriers and bridges to the renewal of ecosystems and institutions, p. 3–37. Columbia University Press, New York.

IFPRI, 2008, ‘The impact of agricultural extension and roads on poverty and consumption growth in fifteen Ethiopian villages’, IFPRI Discussion Paper 00840, International Food Policy Research Institute, Washington, DC, viewed 27 August 2013, from http://www.ifpri.org/sites/default/files/publications/ifpridp00840.pdf

Kahan M.H. & Kahan M.S., 1995, ‘Agricultual growth in China and sub-Saharan African counties’, The Pakistan Development Review 34(4), 429–456.

Leeuwis, C. & Van den Ban, A., 2004, Communication for rural innovation: Rethinking agricultural extension, 3rd edn., Blackwell, Oxford.

Neuchâtel Group, 1999, Common framework on agricultural extension, Neuchâtel Group, Paris, viewed 27 August 2013, from http://www.g-fras.org/fileadmin/UserFiles/Documents/Frames-and-guidelines/New-paradigms/Common-Framework-on-Agricultural-Extension.pdf

NEPAD (New Partnership for African Development), 2003, Comprehensive Africa agricultural development programme, NEPAD, Pretoria.

Oleru, J., 2005, ‘Assessing impact of contract agricultural extension system: Promotion of four priority enterprises in Arua District, north-western Uganda’, African Crop Science Conference Proceedings, vol. 7. p. 733–738.

Owens, T., Hoddinott, J., & Kinsey, B., 2003, ‘The impact of agricultural extension on farm production in resettlement areas of Zimbabwe’, Economic Development and Cultural Change 51(2), 337–357.

Purcell, D.L., & Anderson, J.R., 1997, Agricultural extension and research: Achievements and problems in national systems, World Bank, Washington, DC.

Romani, M., 2003, The impact of extension services in times of crisis: Côte d’Ivoire (1997–2000), Centre for the Study of African Economies, University of Oxford, viewed 27 August 2013, from http://www.econ.yale.edu/conference/neudc03/papers/4a-romani.pdf

Spielman, D., 2008, ‘Encouraging economic growth in Ethiopia: Perspectives on agricultural input markets, agricultural extension and advisory services, and agricultural education and training’, background paper prepared for a study on agriculture and growth in Ethiopia, viewed 27 August 2013, from http://users.ox.ac.uk/~econstd/Brief_DSpielman.pdf

Staatz, J.M. & Dembele, N.N., 2007, Background paper for World Development Report 2008, World Bank, Washington, DC, viewed 27 August 2013, from http://siteresources.worldbank.org/INTWDR2008/Resources/2795087-1191427986785/StaatzJ&DembeleN_AgriForDevtInSSA_ve19.pdf

Temu, A.E., Nyange, D., Mattee, A.Z. & Kashasha, L.K. 2005, Assessing rural services, infrastructure and their impact on agricultural production, marketing and food security in Tanzania, Arusha, Tanzania Development Gateway, viewed 27 August 2013, from http://www.tanzaniagateway.org/docs/AssessingRuralServicesIandInfrastructureonAgriculturalProduction_Tanzania.pdf

Van den Ban, A.W. & Hawkins, H.S., 1996, Agricultural extension, 2nd edn., Blackwell Science, Oxford.

Waddington, H., Birte, S., White, H. & Anderson, J., 2010, ‘The impact of agricultural extension services’, synthetic reviews, SR009 Protocol, International Initiative for Impact Evaluation, New Delhi, viewed 27 August 2013, from http://www.3ieimpact.org/media/filer/2012/05/07/009%20Protocol.pdf

WFP, 2005, Briefing note, African Union Commission Report on Food Security in Africa, AU Summit, Sirte.

Walker, T., Tschirley, D., Low, J., Pequenino Tanque, M., Boughton, D., Payongayong, E. & Weber, M., 2004, ‘Determinants of rural income in Mozambique in 2001–2002’, Research Report 57E, Ministry of Agriculture and Rural Development, Directorate of Economics, Maputo, viewed 27 August 2013, from http://ageconsearch.umn.edu/bitstream/56061/2/wps57E.pdf

White, H., 2009, ‘Some Reflections on Current Debates in Impact Evaluation’, Working Paper 1, International Initiative for Impact Evaluation, New Delhi.

Appendix 1

TABLE 1: Impact evaluation reports consulted for the study, agricultural extension models addressed and impact evaluation results reported by country.

 

Crossref Citations

1. Effectiveness of Extension Services Among Food Crop Farmers in Ekiti State, Nigeria
J. O. Oluwasusi, Y. O. Akanni
Journal of Agricultural & Food Information  vol: 15  issue: 4  first page: 324  year: 2014  
doi: 10.1080/10496505.2014.952175