Last week, we released AidData’s Listening to Leaders report which sought to a answer the question: which development partners do leaders in low- and middle-income countries prefer and why? The report has already attracted far more attention than we expected. It is stirring discussion, debate, and introspection within development partner institutions and also among their external constituencies. We welcome this discussion and hope that this heightened interest facilitates more substantive dialogue between development partners and their in-country counterparts about what is working, what is not, and how their collaboration can be improved.
One of the report’s features that has received a great deal of attention is our ‘Value for Money Index’ which analyzes the relationship between the money development partners spend and their relative influence in policymaking discussions. The Value for Money index is the first attempt to combine data on the policy influence of bilateral and multilateral development partners, as perceived by participants in the 2014 Reform Efforts Survey, with international development finance data from AidData. The Index compares each development partner's reported level of agenda-setting influence in relation to the size of its international development finance commitments. With this information, we can approximate the extent to which a development partner punches above or below their financial weight in shaping the reform agenda in other countries.
Updated Value for Money index in the Listening to Leaders report (AidData)
Over the last week, we have received a great deal of feedback from those who have read the report and closely reviewed the construction of the Value for Money index. In response to this feedback, we have decided to make some small methodological adjustments in how we generate the Value for Money index. These changes are now reflected in an updated version of the Listening to Leaders report available online and we preview them for you here.
First, the updated Value for Money index uses the average annual amount of international development finance committed by each development partner between 2004-2010, rather than total international development finance commitments between 2004-2011. This revised formulation allows us to redress some missing data challenges and generate more accurate estimates of which development partners are punching above and below their financial weight. Second, thanks to feedback from the International Fund for Agricultural Development (IFAD), we identified a coding error in our financial data that has now been corrected and has been properly adjusted in the updated Listening to Leaders report.
So, what do these changes actually mean in practice? The improvements made to the Value for Money index have not significantly changed any of our top-line findings in the Listening to Leaders report. Multilaterals and a select group of small DAC bilaterals are still strong performers, while large DAC bilaterals and non-DAC bilaterals comparatively lag behind. We also see only slight changes in the estimated performance of some individual development partners. For example, the index scores for the World Bank, the African Development Bank, and the United States declined slightly, while those of the IMF, IFAD, and Sweden increased by small margins. All of these updates can now be found in the ‘Performance of Top and Bottom 10 Development Partners by Financial Weight’ table and the ‘Value for Money’ graph on pages 2 and 19 of the Executive Summary. These changes are reflected in Chapter 4 of the full report.
We are tremendously grateful for the constructive feedback that we have received to date from readers of the Listening to Leaders report that have helped us refine our measures of development partner performance. We remain committed to the principle that any measures of development partner performance that AidData generates should be scrutinized, refined, and improved over time.
Now is a particularly opportune time to think critically and broadly about the “art of the possible” for how we can leverage the first-hand insights of leaders from low- and middle-income countries to better assess development partner performance and influence. AidData is currently updating our questionnaire to support a 2016 survey wave and a 2018 survey wave and welcome your feedback. If you’d like to share any insights regarding the current set of performance measures in the Listening to Leaders report or future measures that we could generate with new data or methods, please drop us a line at info@aiddata.org.
