Government investments into climate-smart agriculture nearly doubled last year, as countries responded to growing challenges to global food systems driven by extreme weather events. Two out of every three dollars loaned last year by the World Bank to developing countries for agricultural and food projects were specifically for climate finance. As policymakers dedicate more funding for programs to help farmers adapt, it will be paramount to discover what works—and what doesn’t.
Yet impact evaluations cost money (one survey found 2% or less of a typical program budget is devoted to evaluation) and, as a result, are infrequently done (perhaps less than 10% of projects receive a thorough impact evaluation). How can we leverage new techniques and data sources to evaluate climate-sensitive agricultural interventions with the highest accuracy and lowest cost?
In late January, leading experts in impact evaluations and sustainable agriculture gathered at William & Mary for a workshop to help answer that question. As part of the GeoField initiative, they have worked for the past two years to build a set of use cases that could have wide applications for measuring the impact of climate adaptation investments. Drawing on a rich tapestry of academic and professional backgrounds, event participants distilled insights from their use cases, while also collaborating on new chapters for a forthcoming book Geospatial Impact Evaluation in Practice, which introduces Geospatial Impact Evaluations (GIEs) as an innovative methodology that integrates geospatial technologies and spatial analysis into impact evaluations of development programs.
GeoField, a joint initiative of AidData at William & Mary, DevGlobal, and MercyCorps, was launched in September 2022 with a $4.74-million, four-year investment from the Gates Foundation.
“We need to reduce the barriers to the use of Earth observation—cutting-edge ways to use satellite or other remotely sensed data to understand conditions on the ground—across the full spectrum of agricultural impact evaluations,” said Dr. Ariel BenYishay, AidData’s Chief Economist, one of the Principal Investigators on GeoField use cases, and one of the co-editors for the GIE textbook. “Through our growing community of practice, a highlighted set of use cases, and our in-progress textbook, we aim to build a repository of shared knowledge on how to leverage Earth observation to expand the range of outcomes that can be assessed (particularly environmental ones), improve the cost effectiveness of and rigor of Geospatial Impact Evaluations (GIEs), and enable evaluation of difficult-to-access geographies.”
The workshop, organized with financial support from the Gates Foundation and the Global Research Institute at William & Mary, provided an opportunity for researchers and program implementers to spotlight a diverse and innovative set of GIEs that leverage Earth observation and advanced algorithms.
The GeoField use cases discussed at the workshop exist at the intersection of two rapidly advancing frontiers: the potential of technological advancements and the urgency of climate change adaptation.
Program evaluators commonly encounter major challenges, such as difficulty accessing historical data and measurement errors in self-reported information. How can we gauge the long-term impact of a program when it wasn’t originally designed to measure sustainability? How can we detect changes when treatment and comparison groups weren’t established before the intervention? What innovative methods can we use to overcome these obstacles and ensure that our findings accurately reflect a program’s true impact?
GeoField use cases document valuable insights gained from tackling evaluation challenges in specific implementation contexts, highlighting innovative approaches and practical strategies to unravel complexities in the field. One such example is an emergency crop evaluation program in Ethiopia, implemented by the International Potato Center (CIP) from 2021 to 2023. The program aimed to provide vulnerable households with access to nutritious and resilient varieties of potato and sweet potato, along with agriculture and nutrition training. By leveraging geospatial tools, evaluators identified nearby comparison groups, which were similar but did not receive the program. This enabled the treatment effect (access to sweet potato seeds and agricultural training) to be quantified precisely. Remotely-sensed data was crucial in providing clear measurements that identified the crops grown, which yielded insight into how the new seeds from the program were adopted by local farmers.
“Agriculture is a data-driven enterprise. And yet, unfortunately, in emerging markets, we don't have a lot of data to drive decision making, to prioritize investments, or to shape policy. By bringing in Earth observation, it allows us to monitor trends over time,” said David Bergvinson, Senior Advisor at DevGlobal.
By leveraging widely-available, continuously-operating geospatial data sources, GIEs unlock significant opportunities to transform impact evaluations—making them more cost-effective in difficult-to-access areas, enabling the tracking of long-term sustainable impacts, and allowing even programs not initially designed for precise statistical measurement to assess outcomes like agricultural output and food security.
As an emerging discipline, however, one challenge presented by GIEs is that they require a high level of technical expertise across diverse disciplines. This presents a barrier to entry, one which the Gates Foundation’s support for the GeoField initiative hopes to solve.
“It's an investment to learn how we can better incorporate geospatial tools into the impact evaluation work. I work with probably 15 to 20 different evaluation firms across the world, many of which have already tried to incorporate remote sensing with various degrees of success. So I think we really need to learn how to do it more efficiently and more effectively,” said Richard Caldwell, a Senior Program Officer at the Gates Foundation.
Towards that end, one goal of the workshop was to exchange insights on the work-in-progress GeoField use cases and the GIE textbook, as well as to deepen relationships among key sets of impact evaluators and remote sensing specialists with those in the field.
Presenters of use cases hailed from AidData, Mercy Corps, Tufts University, the Food and Agriculture Organization of the United Nations, the International Rice Research Institute, and the University of Arizona, among others. They spotlighted a diverse and innovative set of ways to incorporate Earth observation technologies in assessing the impact of climate adaptation and agriculture programs.
The presentations covered a wide array of geographies and challenges, addressing topics from agricultural innovations and risk management to food security and environmental strategies. Presenters explored how the use of alternate wetting and drying techniques in Bangladesh, the adoption of orange-fleshed sweet potatoes in Ethiopia, and the strategic use of nitrogen fertilizers and stress-tolerant rice varieties in India are boosting crop yields and reshaping farming practices in diverse climate conditions through effectively using earth observation technologies.
Other use cases showed how Earth observation data can measure the impacts of programs to promote secure water management and sustainable land use, exemplified by canal rehabilitation projects in Somalia and the formalization of land rights in Niger. Another set of use cases highlighted programs to enhance disaster risk management, through initiatives like sugarcane planting in Nepal and drought insurance in Nigeria. This diversity of applications underscores the vital role of using geospatial data in evaluating programs that help mitigate climate-related risks.
“Besides showcasing the versatile applications of incorporating Earth observation technologies in GIEs, the GeoField community is also working towards making these complex methodologies easier to understand and use by a broader audience of researchers, practitioners, and students across various disciplines,” said Dr. Kunwar Singh, a Senior Geospatial Scientist at AidData and one of the co-editors of the forthcoming GIE textbook.
To support this effort, the second day of the GeoField workshop focused on the development of the textbook Geospatial Impact Evaluation in Practice, designed to serve as a practical guide for those looking to engage with GIE methodologies. The lead authors of chapters were drawn from diverse academic backgrounds and experiences, but gathered here for the shared mission to expand knowledge in the field.
“It's been very useful to learn what everybody else is writing about, to make sure that my chapter will be aligned with what they're thinking, and that it will be useful for the students,” said Dr. Leona Ba, a Faculty Fellow at Georgetown University. She has been teaching a course on culturally responsive and equitable evaluation at the Evaluators Institute since 2012 and will contribute a chapter on this topic to the textbook.
In addition to Dr. Ba’s chapter, the textbook will cover topics such as sampling for spatial discontinuity designs, ethical considerations in GIE, and integrating remote sensing and randomized controlled trials, to name a few. The goal is to serve as an in-depth introduction for researchers, students, and practitioners across different sectors to advance their understanding of GIEs and how to use them effectively.
The textbook will be the first on the market with a specific focus on GIEs. Thanks to generous financial support from the French Development Agency, it will be available as an open-access resource, ensuring that its innovative insights reach a global audience.
The presentations of the textbook chapter drafts sparked lively discussions, as peers offered constructive critiques and suggested enhancements based on their own experiences in the field. The collaborative environment not only helped to refine the clarity and rigor of each chapter, but it also ensured that the textbook will be a comprehensive resource, bridging theoretical insights with hands-on practices.
The GeoField community brings together impact evaluators, remote sensing experts, natural scientists, and practitioners to embrace and advance the use of geospatial technologies in impact evaluations. The spirit of this dynamic network promises to drive meaningful progress in how we evaluate, understand, and respond to a rapidly changing world. We look forward to the continued collaboration and enduring impact of GeoField.
For more information about the open-access GIE textbook, GeoField's community of practice, papers in forthcoming special editions of the Journal of Development Economics and the International Journal of Applied Earth Observation and Geoinformation, follow GeoField on LinkedIn today.
