A geospatial impact evaluation (GIE) in Mali, conducted with the German Institute for Development Evaluation (DEval), is aiming to look at how farmers are adapting to climate change and whether small-scale irrigation projects can strengthen the climate resilience of rural communities.
The question carries urgency in Mali, exposed as it is to dry, hot conditions and extreme climatic events like droughts, heavy and irregular rains, and heat waves. The agricultural sector of Mali, with 80 percent of its population engaged in agricultural activities, holds great potential for driving economic growth. But rising temperatures and decreasing rainfall mean more water is needed for agriculture—across the Sahel, including in Mali, agriculture is largely rainfed.
The GIE—results from which won’t be ready until next year—tackles a number of issues: food security, climate change, gender inequality, and conflict. It makes use of diverse datasets and methods, including satellite imagery, geocoded project locations, and geocoded survey data.
As ambitious as the range of development sectors studied were the attempts by the joint team to link the issues, reflecting that climate in Mali affects the social, ecological, economic and political situation of the country. The subject matter of the evaluation was pumps and shallow wells in rural communities in northern Mali, but the lessons touch on whether the interventions wrought higher-level development changes, including sustained social, ecological and economic resilience.
Results of this evaluation will have a real impact in the policymaking arena, contributing to strategies and plans of Germany’s adaptation portfolio. The evaluation will be shared with the Federal Ministry for Economic Cooperation and Development (BMZ) and the Federal Ministry for the Environment, Nature Conservation, Nuclear Safety, and Consumer Protection (BMU), as well as their implementing organizations, the KfW development bank (KfW) and the Gesellschaft für Internationale Zusammenarbeit (GIZ). The findings will also be shared with UN agencies, knowledge organizations, and civil society actors working on climate change, as well as Mali’s local, state, and central government policymakers.
The GIE builds on existing evaluations to test for the first time theoretical expectations of small irrigation infrastructure interventions. A key approach to strengthening climate resilience is adaptation, and so interventions were designed to make rural communities less affected by climatic fluctuations, especially fluctuations in precipitation. And while studying the effect of major irrigation projects, like dams, is common globally, in Mali the AidData researchers looked at small-scale pump-based irrigation, gravitation-based irrigation, and the use of floodplains.
GIEs are still a relatively young set of methods, so it’s unsurprising this particular project featured several innovations. The researchers were able to take a longer view of the projects implemented to date, looking at sustainability among other questions. “Many evaluations of small-scale agriculture have explored the short-term effects of these projects,” said Ariel BenYishay, AidData’s Chief Economist and Director of Research and Evaluation. “We felt that two or four years after project completion isn’t enough to know whether vulnerabilities have permanently been reduced and whether farming communities have become resilient in dealing with future climate shocks and stressors. Adding adaptation to climate change adds an extra layer of complexity, because short-term effects that seem to make a positive contribution might turn out to be inadequate strategies of adapting to future climate conditions and may even lead to maladaptation.”
The AidData researchers also studied the complicated impacts of irrigation measures in the context of Mali’s ongoing armed conflict. Water shortages are one factor in the conflict, and it has been widely documented that conflicts involving access to and use of water are on the rise globally. For this particular evaluation, the AidData team was able to build on their experience conducting a previous GIE on behalf of USAID focusing on the rehabilitation of irrigation infrastructure in Afghanistan.
The team applied a gender lens to the project, studying how women are participating in the irrigation projects and, more broadly, how climate disruptions and conflict are impacting gender inequality.
The Mali study sample consisted of nearly 1,000 distinct locations where small-scale, river- or rainwater-based irrigation had been introduced. The interventions were implemented from 1999-2020 in different regions of the country, including Dogonland, Koulikoro, the inland Niger delta, Sikasso, and Timbuktu. The staggered roll-out of the interventions and repeated observations over 20 years allowed the team to compare the pre- and post-irrigation outcomes of locations, while adjusting for confounding factors. They geospatially linked data on the irrigation projects with agricultural conditions as measured by high-resolution satellite imagery and surveys. A long time series of medium-resolution satellite imagery was used to assess greenness, crop productivity and water availability, while higher resolution images from Google Earth Pro and other sources were incorporated into land-use change analysis and the study of environmental impacts like soil erosion. The researchers drew on large-scale surveys, like the Demographic and Health Surveys (DHS) and the Living Standards Measurement Study (LSMS), to look at child stunting and household incomes, anthropometrics (child height and weight). They also integrated geocoded conflict data from the Armed Conflict Location and Event Data Project (ACLED) and the Social Conflict in Africa Database (SCAD) into their statistical models to better understand whether and to what extent project success is affected by ongoing conflict.
Interest in GIEs will see AidData next conduct trainings with the UN’s Food and Agriculture Organization (FAO), based in Rome, Italy. These will fill knowledge gaps and develop new methodologies for assessing FAO’s project impacts, in particular those as part of the Hand-in-Hand Initiative. Launched in October 2019, Hand-in-Hand is an evidence-based, country-led and country-owned initiative of FAO to accelerate agricultural transformation and sustainable rural development to eradicate poverty (SDG1) and end hunger and all forms of malnutrition (SDG 2), thus also contributing to the attainment of other Sustainable Development Goals (SDGs). FAO sees GIEs as a promising tool to assess the impact of projects at the nexus of agriculture and climate change, notoriously difficult to measure.
“Past evaluations of irrigation interventions in the agricultural sector have focused on outcomes such as food security levels and levels of poverty without making explicit the link to adaptation to climate change,” said AidData Geospatial Scientist Kunwar Singh. “Climate resilience is not itself a directly observable physical phenomena, and so evaluating the impacts of irrigation projects on climate resilience is done by using other observable metrics, such as vegetation and water indices. While some of these, alone or together, have been tested in previous GIEs across geographies, related data on observed and projected climate trends (e.g., temperature and precipitation) are needed to provide an improved measure of climate resilience. For example, datasets representing sub-national seasonal or regional availability of water resources are essential to effective GIEs.”
Securing the supply of water in rural areas in countries like Mali is an important priority of German development cooperation. Germany commits more than 2 billion EUR annually to the effort, making the country one of the most important donors in financing climate change adaptation through development cooperation.
“Precise project location information and high quality satellite imagery played an important role in this evaluation,” said Research Scientist Seth Goodman. “High resolution imagery from Planet and long time series imagery from Landsat enabled us to analyze critical trends before and after project implementation. Project records on the exact locations of interventions—down to the set of plots being irrigated—mean we can look at local impacts in ways that are not possible using approximated data."
DEval played a lead role in the development of the evaluation’s theory of change, worked with AidData on the analysis, and separately conducted focus groups in Mali to better understand the research findings and ‘ground truth’ the evidence uncovered.
The inherent advantages of GIEs proved their worth in Mali, a physically large country where annual data collection on the ground is not feasible, security concerns are high, and the interventions being studied vary across geographical space.
“Our research adds to the growing body of successfully-completed GIEs. It advances existing studies by focusing on longer time frames than most. In addition, it uses a multitude of remotely sensed data sources that allow us to explore multiple outcomes with robust results,” said Senior Research Analyst Rachel Sayers.
“After already having cooperated twice with AidData on its Listening to Leaders Survey, this has been another great experience working with one of the leading research institutes in development cooperation,” said Jörg Faust, Director of DEval. “Beyond providing very useful insights on Germany’s support of climate adaptation in Mali, the GIE also is highly compatible with DEval’s aim to promote rigorous impact evaluations in German Development Cooperation.”
AidData and DEval have conducted joint analysis and presentations of successive waves of AidData's Listening to Leaders Survey of developing country policymakers, resulting in two shared evaluation reports (2016 and 2020) studying how partners in developing and emerging countries assess the support provided by donors, including Germany. This combined effort resulted in DEval's 2020 evaluation report “Development Cooperation from a Partner Perspective,” data from which was also used in the joint DEval-AidData paper “Aid Management, Trust, and Development Policy Influence,” appearing in the peer-reviewed journal Studies in Comparative International Development, as well as a 2021 DEval policy brief.
In addition to BenYishay, Sayers, Goodman and Singh, the AidData team implementing the GIE included Junior Data Analysts Madeleine Walker and Christian Baehr.
DEval, founded in 2012 and overseen by the German government’s Ministry for Economic Cooperation and Development (BMZ), has as its main mission the independent analysis and assessment of German development cooperation interventions, as part of which it conducts a comprehensive and multi-annual evaluation of Germany’s adaptation portfolio.
