Despite the considerable time, money, and effort expended by donors, international organizations, and NGOs to influence policy change in low and middle income countries, there is a lack of understanding about how they can most effectively influence reform efforts on the ground. In a new report launched today, AidData draws upon the firsthand experiences and observations of nearly 6,750 policymakers and practitioners in 126 countries to answer these critical questions.
The Marketplace of Ideas for Policy Change report examines the influence of over 100 external assessments of government performance -- from cross-country benchmarking exercises and watchlists to country-specific diagnostics and conditional aid programs -- on the policymaking process in low and middle income countries. Participants in the survey identified the specific sources of external analysis and advice that were used by key government decision-makers between 2004 and 2013 -- and why. Survey respondents also provided detailed information about reform processes within their own countries, such who has advocated for reform in different sectors and who actively obstructed reform efforts.
“One aspiration of this report is to raise awareness within the policy community that donors and recipients transact in multiple currencies: they transact in dollars and they transact in ideas,” said AidData Co-Executive Director Brad Parks. “If we want to understand the organizational impact of an aid agency, we have to go beyond just measuring the impact of the dollars they spend. We need a better understanding of the extent to which the analysis and the advice they’re putting forward is perceived to be useful, and when these ideas are actually used to inform policy priorities and decisions.”
Some of the key findings from the Marketplace of Ideas for Policy Change report include:
- Despite widespread concerns about external policy advice infringing upon a government’s autonomy, decision-makers in low and middle income countries seem to want more, not less specific policy guidance. The picture that emerges from the survey is not one of governments being cajoled or coerced into pursuing reforms that align with donor priorities and displace domestic priorities, but rather that these governments have significant agency. They are strategic and selective. They pick and choose external performance assessments based on whether they are perceived as helping move the country in a direction that aligns with domestic priorities.
- When survey participants were asked to select from a long list of potential reasons why external performance assessments were influential in their country, the most frequently identified answer was that a given external performance assessment proved influential because it promoted reforms that aligned with the policy priorities of the domestic authorities. External performance assessments are also more influential -- and effective -- when they source their underlying data from the host government.
- External performance assessments seem to help establish a shared diagnosis of policy problems and bridge the gap between the policy priorities of host governments and those of development partners.
- One of the more surprising findings from the report is that country-specific assessments appear to exert greater policy influence than cross-country benchmarking exercises. Country-specific assessments seem to be more popular with policymakers in low and middle income countries because they contain in-depth diagnostic and advisory content. These assessments may also resonate with local stakeholders because they are more attuned to local needs and realities.
- However, when decision-makers in low and middle income countries want to take on complex problems like corruption and institutional dysfunction, they seem to prefer cross-country benchmarking exercises like the UN Millennium Development Goals, the World Bank’s Doing Business Report, and the Millennium Challenge Corporation’s Country Scorecards. This may be because these cross-country assessments provide greater flexibility and maneuverability for policymakers to experiment with different types of reform strategies and iteratively adapt in pursuit of better outcomes.
- Three external performance assessment characteristics are associated with significant increases or decreases in the success of in-country reforms: whether the underlying performance data is sourced from the host government (more successful), whether the assessment is sponsored by international organizations with a global reach (more successful), and whether the assessment focuses on governance reform (less successful).
- Analysis of the survey data also suggests that there is an “influence dividend” associated with assessment familiarity. The most competitive players in the marketplace of policy ideas appear to be those organizations that invest in marketing, communications, and branding.
How do money, ideas and reform efforts come together to produce better development outcomes? The Marketplace of Ideas for Policy Change report begins to shed light on some facets of this question, but more data and further research is needed. To this end, AidData will release a series of publications and blog posts via aiddata.org in 2015 and 2016 that further leverage the data from the 2014 Reform Efforts Survey. AidData’s survey practice is preparing for its next omnibus survey slated for 2016, as well as more frequent and targeted snap polls to take the pulse of the development policymakers and practitioners.
“This report is the result of five years of patient investment by more than 40 different students, staff, and faculty. It began with a simple, but audacious idea in 2010: that we could survey a representative sample of development policymakers and practitioners from every low income and lower-middle income country in the world. This initial flash of inspiration resulted in the creation of a first-of-its-kind sampling frame of nearly 55,000 high-level and mid-level host government officials, development partner staff, civil society and non-governmental organization leaders, private sector representatives, and independent experts in in 126 low income and middle income countries. We believe this sampling frame will provide a strong foundation for follow-up surveys,” said Parks.