About the Event

On October 24th, 2017, AidData brought together a range of actors in the international development space to discuss how donors are missing the mark in targeting aid to reduce poverty and increase per-dollar impact. Beyond the Tyranny of Averages: Development Progress from the Bottom Up—a report from AidData that examines whether and how effectively, bilateral and multilateral donors target aid projects to the most disadvantaged localities within countries—served as a basis for the conversation. Panelists offered their perspectives on the challenges of detecting hotspots of poverty and marshaling resources more equitably to ensure that no one is left behind.

Panelists

Selim Jahan: Director of the Human Development Reports Office at the United Nations Development Programme

Amanda Glassman: Chief Operating Officer and Senior Fellow at the Center for Global Development

Caroline Heider: Director General and Senior Vice President for Independent Evaluation at the World Bank

Kevin Croke: Economist at the Development Research Group of the World Bank

Brad Parks: Executive Director of AidData

Moderator

Samantha Custer: Director of Policy Analysis at AidData

Three Top-line Takeaways from Beyond the Tyranny of Averages

1. There’s evidence that donors prioritize efficiency over equity in where they choose to allocate funds at the subnational levels—they favor wealthier areas with good infrastructure and more people that can be positively affected.
2. Political expediency plays a role in allocation; Chinese assistance disproportionately flows to birth places of leaders, while this is not the same of World Bank.
3. Aid has mixed results on development outcomes at local levels; some types of aid may worsen local governments and have unintended effects on corruption, violence, etc.

Introductions & Discussion

Some would argue “leave no one behind” is too aspirational by 2030; what needs to change in order to turn this rhetoric into reality?

Selim: “No One Left Behind—you can think of it as an aspiration, but in order to achieve something you need aspiration” (06:50)

• We need a proper mapping of who is left behind and what they need
• Once we know this, then we can decide what actions to take
• Inequality is a market condition
• External resources should supplement, not be prioritized over, domestic efforts; priorities, strategies, and resources must be owned by the developing countries, rather than placing external efforts in the driver’s seat of domestic growth

Amanda: In that a lot of her experience has been within the global health policy arena, she admires AidData and its BTA report because it "identifies resource allocation as the primary lever that we have to affect change” (14:32)

• In global health, this has not always been in the center of attention; the field mostly focuses on between-country allocation, although the issue really is within-country allocation
• “When you look at global health centers … where needs are highest, you see a complete mismatch between the allocation of resources within countries and the needs as they’re identified”
• How are we allocating resources now—both government and aid resources? It depends on the goal: is it redistribution in global health, Is it health maximization, is it votes?
• There are always multiple goals at play when it comes to aid and domestic spending allocation
• Aid often follows government spending; advice to look at domestic expenditure at the same time as aid allocation--how are these two working together or working separately?

Caroline: “How can we translate that past experience into insights that are valuable for the future?” (20:00)

• 3 models that World Bank uses to ensure that findings from evaluation get absorbed into future policy programming
• Do an evaluation before a new policy is adopted
• Series of strategies adopted to show how well the past stacks up to these new aspirational targets
• Allow a certain period of time for a new strategy to be implemented, then look at whether things are on the right track to deliver on the results that the strategy promised
• Relates directly to World Bank’s efforts to reduce poverty and boost shared prosperity; “leaving no one behind”
• With the SDGs and their complexity, there will be a paradigm shift and the international development community will need completely new solutions
• Evaluation on what World Bank has done to address data for development
• WB has been very good at generating data and analyzing it; but in terms of work, how do you stimulate the use of data?
• A lot of decision makers don’t understand what data tells them and what they need to do in terms of making decisions based off of it
• It’s important to have more data and make it more granular for within-country distribution, but we need to be mindful of what the data tells us, and--echoing Selim--think about the best pathways to actually help reduce poverty and inequality.
• Let’s look at what the key barriers actually are and use data to remove them

Samantha: “The data is wonderful, but is it making decision making easier, is it being used effectively?” (28:25)

Kevin: sits in World Bank’s Development Impact Evaluation Unit in the Research Group; actually uses geospatial data in that he does evaluations of global development outcomes (29:30)

• In thinking about what the new sources of data have helped us do as well as their strengths and limitations, refer to two sectors that he has experiencing using this data in:
• Applied health impact evaluations and analyzing the drivers of health decision making
• Geotagging
• What would be more effective is higher frequency and resolution; satellites won’t capture this, so in the health space, what could we do to have increased spatial and temporal resolution?
• Large infrastructure projects, many of which in the transport sector
• Remote sensing data sources, nighttime luminosity measures, environmental forest and land cover; all described in the report, these measures have opened up a new way of doing impact evaluations of large infrastructure projects
• What we get with the geospatial tools is a binary ‘yes’ or ‘no’ (did it work or did it not work?). However, we still need to work with policymakers regarding what combination of activities will work best and then implementing a series of evaluations to test this synthesis.

Samantha (to Brad): “What do you see as the current state-of-the-art in terms of making the connection points between resources, results, decisions, and data, and where do you see the field moving in order to trigger this data revolution that Kevin mentioned?”

Brad: Cites a promising work by Marshall Burke and David Lobell from Stanford University (38:40) that is able to detect welfare changes amongst the very poorest of the poor as an extremely important step in ensuring that no one is left behind (37:30)

• Burke and Lobell’s work uses daytime satellite imagery and training an algorithm with household surveys to figure out what daytime satellite imagery features correlate with changes in development outcomes
• Talks about nighttime light satellite imagery features that correlate with development outcomes; if we’re trying to more effectively evaluate development outcomes with an eye of leaving no one behind, this is extremely important especially to detect progress for the poorest of the poor
• We must let go of fantasy that governments/aid organizations are going to solve the NOLB measurement by putting out more/more frequent gold standard surveys
• We need to get smarter about tracking development progress; combining strengths of household surveys with the temporal frequency and spatial resolutions of satellites
• Let’s think about smart investments to combine proven methods—don’t focus too much on one kind (especially if they’re expensive like surveys) to develop an effective no one left behind tracking

Samantha: “We see the size of the inequality challenge and we know the imperative to leave no one behind, but we absolutely need to get smarter in how we detect these inequalities, how we measure progress, whether it’s in terms of resource allocation or improvements in local development outcomes, and ultimately we need to ensure that this data and evidence that we already have is actually being effectively used by policymakers both globally but also nationally.”

Questions

Brian Bingham—works with USAID’s Global Development Lab in Center for Development Research

• AidData has been focused on research translation—helping government, donors, implementing partners, etc. to understand/use this satellite data to make decisions based off of it.
• At some point the investment community must help government, other donors, and implementing partners understand how to use the data and help policymakers to translate this satellite survey data into something they can actually use to make decisions. When will this support for research translation occur? (43:30)

Zach Christensen—Development Initiatives

• To really ensure that no one is left behind, does there need to be more grappling with data on the ground versus looking down from space? (45:20)

Paulina Migalska—Akhandataa

• In addition to using existing data sources, think creatively about using non-official or unofficial data sources that are already out there--many of which produced by telecommunications companies, private sector actors etc.—that the development space not readily using to the extent that we should. (47:30)

Responses

Selim: In regards to data literacy, policy makers and decision makers within developing countries need the help from the international community in order to become more fluid and empowered in utilizing data (50:30)

• “We can think of data, we can think of evaluation and monitoring, but I think it is also important to think about the capacities at the country level because we have to depend on them later” (54:10)
• Specifically in regards to leaving no one behind, quality of data is just as important as quantity of data
• Even though schools might be in every part of a developing country, is the schooling and curriculum in rural areas up to par with urban areas? If not, that is an indication of accelerating inequality.

Caroline: at IEG, for large evaluations with recommendations, there is a follow-up process once a year for 4 years to analyze what actions have been taken in response to the recommendations given (56:30)

• “Learning engagements”: WB invites counterparts on the operational side of WB to educate them on what is working—and not working—in particular areas based on the data; this, in addition to increasing access to data through short blog posts, papers, etc., works to prevent data from landing in the “data graveyard” and promotes active approaches to utilizing it
• Recommends relying on multiple sources of data to ensure that you’re not missing something; there’s different dimensions in which relying on only one source of data wouldn’t work, so bringing in different types of data will better contribute to a holistic picture

Amanda: “The problem that we have right now is that we’re starting with data and evidence instead of ‘what are the structural mechanisms by which governments and donors allocate and check their money’.” (1:03:26)

• We should start with the latter mechanism; in many countries, the censuses are out of date and within-country decision makers don’t use any—and/or are unaware of—worthwhile evidence
• Strongly recommends more systematically supporting the national statistical offices and administrative data to yield more reliable basic data; “let’s fund to get the basics done!” (1:06:40)
• We need to combine the “gadgets” with these more basic support mechanisms on the ground

Kevin: “if you really want takeup in capacity, you have to spend money on these longer-term investments” (1:12:00)

• Combining methodologies that are technically feasible using the geospatial data results in a much better evaluation than you would implement only using one or the other
• Having within-country project implementation units involved might result in the project moving slower but with the ministry involved

Questions

Paul Cadario--University of Toronto

• “Isn’t one of the questions ‘what are we doing this for’?” What lever will have turned on the most lights at the end of the day? Relating to economic efficiency—where do you employ resources? (1:13:20)

Arya Grabowski--Oxfam America

• Is part of the problem with use that we haven’t made enough of a case for why the data is valuable and why it is capable of leaving no one behind? What can we do to increase the political capital behind collecting and using data? (1:16:05)

Responses

Brad: On board with Amanda and Kevin; satellite methodologies and on-the-ground measurements are generally complements rather than substitutes

• “Having this kind of independent measurement system in the presence of weak national statistical systems is extremely valuable” (1:18:06)
• “A lot of these measurement innovations have happened contemporaneously with the shift in the development policy community towards results-based financing”
• These things are not independent
• Having an independent measure of wealth counters the in-country administrative data that often can be manipulated; example: country governments manipulating their GNI per capita data to stay underneath the IDA eligibility threshold, preventing a reduction in ODA
• Geoquery allows you to spatially join investment data, outcome data, contextual data, etc.
• A lot of this geospatial data doesn’t just tell us about the average impact of these interventions; we can figure out which types of interventions are yielding the biggest results

Caroline: Ultimately, if our success criteria is “luminosity”, this is a dangerous path

• This would undermine what the SDGs are about; in order to determine where we should invest, we must refer to the SDGs, their interrelationship, and how this applies to different country needs
• Developing countries’ needs are not monotonous and each have different challenges; some demand greater access to electricity, to water, resources, etc.

Amanda: Addressing the issue of why data transparency matters and helps

• All organizations should be using the data more to analyze the programs within developing countries; using the data in a more systematic way will yield better results
• Transparency is a big political bonus; data works well to tell good and bad stories
• “The idea is to try and tell a little bit of both so that transparency doesn’t just come out as a liability for organizations” (1:26:38)

Kevin: “Finding measurement initiatives and uses for data that the actors on the ground actually care about and starting from the problems that they identify as opposed to the data that we can collect is really, really important for generating uptake” (1:26:50)

• Basically, thinking about what the resource allocation mechanism is first and then working backwards through the data

Samantha: Cross-cutting theme across both the supply side: opening up more and better this data to address issue of NOLB, and the demand side: once data is available, how do we actually put it to use for decision making

• Relates back to idea of value proposition; all of us think in cost-benefit terms
• To developing country governments, there are major risks to opening up information; what if the data is misconstrued? What if people start finding things that are difficult for us to see and act upon? So how do we increase the perceived benefits of opening up information?
• One way, as third parties, is to be transparent in putting out imperfect information; this invites interest among official sources of data to correct or augment the record
• Big question: When it comes to use, it is not enough to just have done the analysis of geospatial data or survey information; we must ask “how is this adding value in a way that we couldn’t have done before this information was available, and is it worth the pain of me having to understand new information in a new form?” (1:34:25)

This is a challenge AidData faces and an impetus behind the creation of the BTA report