Information is “never the hero”, but it plays a supporting role in how leaders allocate scarce resources and accelerate development in their communities. Even in low- and middle-income countries, decision-makers have ample choices in sourcing evidence from a growing field of domestic and international data providers. However, more information is not necessarily better if it misses the mark for what leaders need to monitor their country’s progress. Claims that information is the “world’s most valuable resource” and calls for a “data revolution” will ring hollow if we can’t decode what leaders actually use — and why.
In a new report, Decoding Data Use: How leaders source data and use it to accelerate development, AidData reveals what 3500 leaders from 126 countries have to say about the types of data or analysis they use, from what sources, and for which purposes in the context of their work. We analyze responses to AidData’s 2017 Listening to Leaders (LTL) Survey to offer insights to help funders, producers, advocates, and infomediaries of development data understand how to position themselves for greater impact.
What information do leaders use — and for what purpose(s)?
Recent calls from the World Bank, OECD, and others to strengthen national statistical systems appear to be well-founded. Not only do the vast majority of leaders utilize national statistics (81 percent), they also rate this evidence to be the most helpful type of raw data produced by domestic organizations. Funders of data and statistics would do well to ramp up support in this area, as we have previously found that demand for national statistics often outstrips its supply due to various financial, technical, and political constraints.
Proponents of rigorous impact evaluations (e.g., CGD, 3ie, AidData) will be cheered to know that international development agencies are not the only prospective users of this information. Domestic leaders most frequently reported using project evaluation data (73 percent) and rated it as most helpful out of all types of evidence produced by international organizations. Development partners should take note: investing in strong evaluation systems not only improves project performance, but helps in-country actors internalize what works, what doesn’t, and why as they deliver assistance.
Overall, many leaders still show a preference for using data to conduct retrospective analyses of past performance rather than inform future policy and programs. However, something appears to shift in the decision calculus for leaders when it comes to how they use the information they deemed to be “most helpful” (on a scale of 0 to 4). Leaders use this class of “most helpful” data at higher rates to carry out forward-looking tasks — such as identifying priorities and selecting implementation strategies — than other data.
Whose information do leaders find most helpful?
Leaders give government agencies the highest marks among domestic information providers. Comparatively, unofficial data sources are underutilized. Only 37 percent of respondents reported using private sector data or analysis, but 80 percent of those who did found it to be helpful. Meanwhile, leaders from politically free countries were more likely than those from restrictive countries to rate CSO data as helpful in their work. In this respect, protecting civic space is not only important from a human rights perspective, but likely also critical for evidence-based decision making.
Multilateral organizations are the preferred providers among international information sources in both overall use and perceived helpfulness. Large multilaterals with broad mandates such as the World Bank, European Union, and the International Monetary Fund are efficiently converting large development assistance budgets into greater-than-expected uptake of their data and analysis. Large bilateral aid providers such as the United States and Germany also perform well on this measure of uptake per assistance dollar.
That said, financial clout is not deterministic in terms of the value that leaders derive from a particular information source. In the figure below, we compare the reported helpfulness of each development partner’s information with their predicted performance based upon the sheer size of their official financial contributions alone. Notably, focused multilateral organizations (by region or sector) with smaller aid budgets such as the the International Fund for Agriculture Development and the Global Fund punch above their weight in terms of the helpfulness of their data and analysis.
Value for Money: Who punches above or below their financial weight in terms of perceived helpfulness of their data or analysis?
Development leaders are less likely to consult other international information sources, though some organizations garnered above-average marks in the perceived helpfulness of their data or analysis. Among private foundations, the Open Society Foundation (92 percent) and the Bill and Melinda Gates Foundation (87 percent) performed best. World Vision leads among implementing organizations (90 percent) and Transparency International comes out ahead of other advocacy groups (83 percent) by a slim margin.
Why are some sources of information more helpful than others?
Survey participants selected up to three characteristics that explained why a given provider’s data or analysis was helpful. On this basis, we preview three recommendations for producers, funders, and advocates of data and statistics that float to the top:
- Context is key. To capture the attention of leaders, information providers must demonstrate a clear understanding of local realities in LICs and MICs
- Be constructive. To motivate leaders to take action, information providers should not only diagnose problems but offer practical policy recommendations
- Know your niche. Leaders expect different things from domestic and international information providers, which is an opportunity for greater specialization
To learn more about the full slate of findings and recommendations from this research, read the Decoding Data Use executive summary and full report.
