In a new report launched last month with the Governance Data Alliance (GDA), AidData presented the results of a 2016 snap poll that asked 3,000+ public, private, and civil society leaders from 126 low- and middle-income countries to share their views on the use and usefulness of governance data in their work. Two insights rose above the rest: world leaders want governance data to be more transparent and context-specific. It turns out that principles of transparency, participation, and collaboration are not only important to advancing open government more generally, but may substantially shape whether and how governance data is put to use.
In recent years, there has been a proliferation of rankings, indices, and assessments of country performance on governance issues such as corruption and transparency. Given the costs involved, producers want to understand whether and how governance data is being put to use to support reform champions, inform policy changes, and improve governance in countries around the world. Simply put, they wish to know the return on their investment. In addition, they are also interested in finding out how best to reach their audiences.
Of the 3,000+ snap poll recipients, 514 leaders (17.6%) reported on which governance data sources they use, how important and helpful this information is in their work, and the specific attributes of the data that are more or less useful to them. In this blog post, we preview a few highlights from the snap poll results featured at length in our new report, In the Eye of the Beholder: When is governance data "good enough"?.
Which governance data is perceived as most useful and why?
World leaders perceive some governance data sources more favorably than others (see Figures 1 and 2). Among the best performing governance data sources are the World Bank Group’s Public Expenditure and Financial Accountability, its Doing Business report, and the International Budget Partnership’s Open Budget Index. Roughly 80% of snap poll participants perceive these assessments to be important and helpful in their work.
Figure 1: Ranking of Governance Assessments by Perceived Importance
Figure 2: Ranking of Governance Assessments by Perceived Helpfulness
This raises a critical question: Why do some governance data sources rate higher than others? Our snap poll participants emphasize two key attributes as differentiating governance data sources that are particularly useful: relevance and credibility (see Figure 3).
Figure 3: Why do leaders use (or not use) governance data?
Leaders view governance data as relevant when it reflects an understanding of the local context, generates new insights, and proposes concrete policy recommendations. And they view governance data as credible when these data producers are transparent in their methods and assumptions (see Figure 4).
Figure 4: Why do leaders find some governance data irrelevant or lacking in credibility?
How should governance data producers respond?
Is governance data good enough? The answer ultimately depends on whether this data is viewed as contextually appropriate and sufficiently transparent by those who make or shape governance reforms in their countries. For governance data producers seeking to “close the feedback loop,” this implies working towards (1) identifying ways to make the substance of their assessments more actionable and specific to realities at the country level, and (2) creating greater openness in how they evaluate country performance.
Increase relevance through more context-specific data
When it comes to improving the relevance of governance data, the burden of ensuring that data is context-specific need not fall on producers alone. AidData’s past research signals that governance data is more influential when producers draw upon local knowledge from resident experts or citizens, involve domestic actors in the data collection process, and include actionable content by pairing problem diagnosis with policy solutions.
Data producers therefore have every incentive to encourage greater participation of, and collaboration with, domestic partners to increase the likelihood that governance data is put to good use. This includes engagement along the entire continuum of data production from collection to identifying recommendations for country-specific reforms.
However, producers may face trade-offs in comparability and global standards when they adopt a greater emphasis on context-specific governance data. Data producers are in the business of assessing the performance of countries on various indicators of good governance. These assessments, in turn, are often based upon some standard or benchmark of what distinguishes “bad” and “good” performance.
This raises questions about what happens when local knowledge or country context flies in the face of advancing global norms of good governance. How should governance data producers respond when local definitions of democracy are at odds with a common global standard? To what extent can country specific assessments be reconciled with the desire among many data producers to create a race-to-the-top dynamic through cross-country comparisons that “name and shame” lagging performers?
The challenge for governance data producers will be to find creative ways to respond to this clear demand from policy makers for data that more directly speaks into their country context, while retaining the ability to inform action and monitor progress against common global benchmarks.
Improve credibility through transparency
Regarding why some governance data is perceived to lack credibility, snap poll participants identified two reasons: (1) opaque methods and assumptions in how the data is created, and (2) concerns that resulting data was untrustworthy or biased. If governance data producers proactively provided users with detailed information on the methodology used to create their assessments, this would not only address concerns regarding opacity, but may also generate a trust dividend as users view transparent producers as more trustworthy than other producers.
However, it is important to emphasize that snap poll participants are speaking to their perceptions of the relative transparency of governance data sources with which they are familiar. This implies that it may not be enough for producers to engage in token transparency, such as publishing their methodologies on their websites, but to continually look for ways to inform and engage their user base around how they define, measure, assess, and validate governance performance in low- and middle-income countries.
As the international community embarks on a major effort to collect data and monitor progress against the sustainable development goals, including SDG16 (peace, justice, and strong institutions), there is no better time for producers, funders, and users to come together to ensure that governance data is well positioned to support both global monitoring and national reforms.
