Policy research and advocacy groups in low- and middle-income countries work tirelessly to inform and shape policies, local to global. Yet even as these organizations aspire to be responsive and relevant to policymakers, they often lack the tools, metrics, or capacity to assess how their efforts are perceived and received by the government officials and other policy actors they seek to influence.
To address this challenge, AidData has partnered with the African Population Health and Research Center (APHRC), one of the continent’s premier think tanks based in Nairobi, Kenya, to test how policy research organizations in Africa can use semi-standardized tools to scale up the collection of important feedback data from their stakeholders in a sustainable way.
Supported by the Hewlett Foundation, this pilot project involved close collaboration between our organizations to field APHRC’s first-ever organization-wide client perception survey. We worked together to determine organizational goals and key indicators, define a target audience and construct a sampling frame, co-design and field the survey, and analyze the results. This provided APHRC with actionable insights, including that more than 60% of stakeholders indicated they want research to be designed with policy impact in mind and involve policy actors from the start.
APHRC, which works across 30 sub-Saharan African countries, seeks to generate evidence that drives policy action improving the health and wellbeing of African people. “Feedback on the way we interact with stakeholders, our performance, and our impact is critical for our work to transform the lives of Africans in Africa,” said Patterson Siema, Director of Policy Engagement and Communications at APHRC. “It helps us understand what has changed as a result of our engagement and what we need to improve to help transform lives through research and evidence. Ultimately, the best validation of our work is to see our research recommendations being adopted in the policy process.”
“The AidData-APHRC partnership demonstrates that with a modest level of financial support and capacity building, organizations can gain the skills and confidence to continuously seek and act upon feedback from their constituents,” said Samantha Custer, Director of Policy Analysis at AidData.“ This helps organizations like APHRC not only understand whether they are successful in their goals but also ensure their financial sustainability: organizations can attract more funding for their work when they demonstrate that their work is used, appreciated, and valued by those they seek to influence,” continued Custer.
What stands in the way of soliciting feedback?
“Aside from funding and capacity, one constraint we faced was the lack of a centralized database of stakeholders that could be readily deployed for a survey or a similar tool,” said Siema. This challenge is by no means unique to APHRC—organizations often struggle with building and maintaining a list of contacts that they view as their key target audiences.
APHRC began by defining their target audience and took a project-level approach to collect information on the various stakeholders. They worked with relevant project and program units to collect information on individuals that worked with them on any project over the previous three years. While time-consuming, this exercise resulted in a database that has become an asset for APHRC, as it serves as a one-stop shop to obtain information on all of APHRC’s stakeholders.
Insights from a client satisfaction survey
In the past, APHRC has collected project-level feedback from stakeholders; however, a client perception survey at an organization-wide level was a first. While the survey produced a wealth of rich information, several findings stood out.
First, when asked how APHRC can best inform policies through its research, a strong majority of stakeholders (over 60%) said they preferred research to be designed with policy impact in mind by involving policy actors when initially developing research questions.
Second, a similar majority indicated a preference for receiving evidence in documents that were short, easy to read and understand. Improved messaging to stakeholders could focus more on bite-sized communication, such as blogs, press releases, media articles, and social media. The same amount of respondents placed a high priority on cultivating relationships with policymakers by supporting them beyond just the dissemination of research. These findings are not unique to APHRC—African leaders who responded to the survey highlighted them as the most important factors for whether data and research is able to inform and impact policy.
Third, respondents provided the names of other organizations whose evidence they found credible. This provides insights into the right partners that can enhance APHRC’s credibility and with whom they can pursue joint activities to increase the uptake of their data and evidence.
For APHRC, the feedback received provides input for their strategic plan, informs a redesign of the Center’s work, and serves as a guide to building and deepening relationships with stakeholders. The consequences are not trivial—the actions APHRC takes as a result of the survey could lead to increased funding, better project results, and meeting organization-wide objectives.
We found that for the feedback process to provide value, it needs to be carried out regularly. This requires staff time and money, but organizations can start with simple and cost effective open-source survey tools. Organizations would do well to build in the costs of such an exercise into project proposals or set aside a small amount of core funding for this purpose. The process would also benefit from a continuity and consistency in the survey content, so progress on certain indicators can be tracked overtime.
Going forward, AidData will explore whether and how semi-standardized surveys can help African policy research organizations like APHRC assess the impact of their work. As part of AidData’s ongoing work to build partnerships with these organizations and bolster engagement with policymakers in Africa, AidData will continue to seek innovative ways to engage with organizations interested in better defining and measuring their policy influence. APHRC is also keen to explore ways to partner with other organizations in the region to help them carry out similar feedback exercises. This will help foster peer learning among organizations with similar mandates and target audiences and help mainstream the practice of collecting performance feedback.
