Ghana, a country of 32 million in West Africa, struggles with widening regional inequality, agricultural decline, and natural resource depletion. Two-thirds of Ghanaians lack access to safe drinking water and only 1.6% of irrigable land in the country is equipped for irrigation. What’s more, a majority of this water-management infrastructure is concentrated in the richer regions in the center and south.
Climate change is worsening these patterns of spatial inequality to produce stark effects: Ghanaian youth are increasingly migrating out of poorer northern areas, as water stress due to changing rainfall patterns makes farming untenable. In the latest census for which data is available, more than two-thirds of all internal migrants within Ghana were leaving from the northern regions.
To combat this issue, Ghana’s government devised a plan—One Village, One Dam (1V1D), implemented under the Infrastructure for Poverty Eradication Programme (IPEP)—to build one new, small-earth dam in every village in the five northern regions of the country. As of 2022, the 500 promised dams are at or near completion. But did the program accomplish its goals?
Alongside the local Ghana Center for Democratic Development (CDD-Ghana), AidData’s Research and Evaluation Unit (REU) is conducting an evaluation of the effectiveness of the 1V1D. Earlier this year, three researchers who lead AidData’s Gender Equity in Development Initiative—Rachel Sayers, Jessica Wells, and Katherine Nolan—traveled to field sites, began evaluation surveys, and worked to pilot a new methodology to better disaggregate geospatial datasets by gender.
Funded through a one-year grant from the William & Flora Hewlett Foundation, their work has the aim of bringing AidData’s geospatial impact evaluations (GIEs) to scale. The team has also received an add-on grant from Innovations for Poverty Action (IPA), with the specific goal of identifying the measurement error in self-reported agricultural characteristics through the collection of gender-disaggregated geospatial data.
“We’re interested to see what the impacts of these dams are more broadly, but also specifically, what the impacts are when disaggregated by gender. This is one of AidData’s first evaluations being conducted with an explicit gendered lens at the outset—it’s our driving focus,” explained Jessica Wells, a Senior Program Manager.
To plan for the analysis, the evaluation team went back to the drawing board. “We asked ourselves what we needed in order to do this well, and how we could push the boundaries of gender GIEs in different types of contexts to create new methodologies that can then be expanded and applied to other sectors,” said Wells.
“The 1V1D project was originally planned to roll out to more communities. We’re interested in understanding: are there factors correlated to receiving a dam or not; what were the impacts of receiving a dam; and how much does the quality of the dam affect those impacts?” asked Dr. Rachel Sayers, a Research Scientist.
The research team also looked at a slew of other key research questions: Does having access to water extend the harvest season, and by how much? Do dams have different impacts for the livelihoods of men versus women, by changing their decision-making or power dynamics? If women are predominantly collecting water from the dam, and that water source is closer, does that free up any of their time and what do they do with it?
To address these questions and more, three facets of data collection were planned. First, survey interviews were conducted with the village leader in each of the project villages. The evaluation team asked about dam characteristics and patterns of use, while a trained enumerator inspected the dam, performed a perimeter walk, and took GPS coordinates. Matching up the GPS coordinates from the dam walk with satellite imagery allowed the team to understand exactly how much water was in the dam and when.
Then, this methodology was repeated with local couples in each of the project villages. Men and women were each surveyed about their plots of land to capture their agricultural use, crop types, irrigation structure, and asset ownership. “We were especially trying to understand what their days look like and how tasks were divided. How do they make decisions about the plots of land? How many hours a day do they spend collecting water?” explained Katherine Nolan, a Research Scientist. By combining the data gleaned from interview questions, dam observations, and satellite imagery, the research team could paint a more complete picture of the outcomes of this project.
Yet, survey interviews can only provide so much reliable data, as interviewees often feel the need to give socially desirable answers rather than accurate ones. This is a problem for evaluators, as econometric models are based on the assumption that measurement error is random. If a farmer is asked about harvest amounts but doesn't exactly remember, the measurement will be off—but unlikely to affect estimates, as the error across all surveys will likely be symmetric. However, if spouses do not know how much the other harvested because of deliberate hiding, then the measurement error will not be random but instead biased. Unless a researcher can correct for it up front, this kind of measurement error could bias an entire econometric model.
“We were particularly interested in understanding how outcomes of the 1V1D program differed between men and women, especially related to household dynamics like income hiding and household bargaining,” said Wells. “As a sanity check on the survey responses, we played a series of behavioral games with a subset of the individuals who answered the survey, so we could ground-truth those results.”
Behavioral games are situations in which individuals have to allocate resources between themselves and other players, while also taking into account the objectives of other players. “These “lab-in-the-field” experiments are designed to create an environment where participants are incentivized to tell the truth, because the stakes are directly related to the outcome of the behavioral game. The game is also intentionally designed to measure an outcome in which you’re interested, albeit indirectly,” said Sayers.
The evaluation team knew from previous literature that when husbands and wives are separately asked about who makes a decision about an issue, their answers frequently do not match up. Both participants might say “I am the sole decision maker,” or one might say “my spouse makes all the decisions” while the other spouse says “we both make the decisions.”
To understand exactly who held decision-making power, the research team designed an experiment where both husbands and wives were separated and asked separately how they would split a specific amount of the local currency (cedi). Couples were then put together and asked to make the decision jointly. “This game tells us what individuals would do if they had sole control and what they actually do in real-life scenarios. These results then help us uncover who in the household holds more decision-making power on income,” said Nolan.
A second behavioral game was played to understand how willing interviewees were to pay in order to keep income from a spouse. In this “willingness-to-pay” game, a random sample of women were offered a certain amount of money over several rounds that they could keep for themselves or give to their spouses. As the game progressed, the amount offered to participants remained the same, but the amounts that could be for spouses (if the option to give the money to the spouse was chosen) increased. At lower amounts, most individuals chose the money for themselves—but as the amount that could be for the spouse increased, the researchers could see what amount was required for a participant to switch to giving income to their spouse.
“That amount indicates how much she values controlling her income or concealing it from her spouse,” said Wells. “This tells us if there is a propensity for her to want to keep money for herself, even if it’s a smaller amount, rather than giving a larger amount to her spouse.”
Finally, to ensure that the behavioral games being played elicited realistic outcomes for accurate data collection, the team randomly selected one of the games to be paid out.
Combining the results from the behavioral games together with the survey data and satellite imagery provided the research team with a new level of evidence often unavailable in impact evaluations. This gave the research team a much-needed additional layer of insight into how measurement error by gender in geospatial data affected a real-world evaluation.
“The combination of self-reported agricultural plot characteristics and remotely sensed measures of these same characteristics provide us key insights into measurement error in agricultural surveys, such as: what is the measurement error in a man’s versus a woman’s reporting over the past year? How much measurement error is there when an individual is asked to report about their own plots versus their spouse’s plots?” added Sayers.
Now that the initial data collection has been completed, AidData and CDD-Ghana are synthesizing data sources, performing comparative geospatial analysis, and extrapolating much-needed information on the measurement error found in gender-disaggregated survey data. After this pilot project is complete, the team hopes to continue scaling their innovative evaluation methodology alongside regional partners in sub-Saharan Africa and beyond to answer future research questions on gender equity, the implications of climate change, and more.
