Tracking Underreported Financial Flows Methodology

AidData's Tracking Underreported Financial Flows (TUFF) methodology provides a systematic, transparent and replicable way of tracking aid and other forms of state financing from non-Western governments—such as China, Saudi Arabia, and Qatar—that do not publish comprehensive or detailed information about their overseas activities.  

How are these TUFF-based data collected, and what steps does AidData take to ensure their accuracy and comprehensiveness? This page provides an overview of the 1.3 version of the TUFF methodology. For detailed information on the methodology, please refer to AidData's TUFF Methodology, Version 1.3.  You can also review the Coder Instructions and Frequently Asked Questions for additional information.  

AidData’s TUFF methodology was designed to provide comprehensive, detailed, and accurate information about projects financed by donors and lenders that do not participate in global reporting systems, such as the International Aid Transparency Initiative (IATI) and the OECD’s Creditor Reporting System (CRS). The TUFF methodology rigorously standardizes and synthesizes information from thousands of available sources to achieve this goal. The methodology is divided into three stages:

Stage One: Project Identification

The objective of Stage One is to identify potential projects through extensive searching of official sources and media reports. Projects undertaken in a particular country and supported by a specific supplier of official finance—be it a sovereign government, multilateral institution, non-governmental organization, or private foundation—are identified through four major sources: (1) aid information management systems (AIMS) in recipient countries; (2) Chinese Embassy and Economic and Commercial Counselor websites; (3) IMF staff country reports; and (4) Factiva, a Dow Jones-owned media database that draws on approximately 33,000 media outlets worldwide in 23 languages, most of which are newspapers, radio and television transcripts.  

Stage Two: Source Triangulation

The objective of Stage Two is to search for and synthesize additional sources for each project identified in Stage One. Research assistants use Google and other search engines, such as Baidu, to confirm or refute a project’s existence and refine the accuracy of a project record. These searches uncover critical details on the project, such as: financing agencies, the donor’s financial contribution, and the status of the project.

During these searches, research assistants can uncover additional sources, including (a) media reports; (b) other donor or recipient government documents—e.g., loan agreements, budget documents, project documentation; (c) country reports published by the IMF, World Bank, the African Development Bank, the Asian Development Bank, and other intergovernmental organizations; (d) information from the websites of contractors responsible for implementing officially-financed projects; (e) multimedia evidence of project activities—e.g., photos and videos; (f) field reports published by in-country NGOs; and (g) academic articles. If any additional projects are identified from these new sources, such projects revert to Stage One before they undergo the source triangulation process in Stage Two. In addition, information gaps can be corrected via correspondence with donor and recipient agency personnel.

Stage Three: Quality Control

The final stage of the process is a series of rigorous, systematic quality assurance procedures.  Using open source information has many potential pitfalls, and we seek to identify and eliminate potential errors, biases, and data gaps. This quality assurance includes the following procedures: identifying and eliminating duplicate records through a thorough recipient-sector review process; reducing double counting by separately identifying umbrella projects and their subsidiary activities; checking the consistency of project categorizations at the record level; careful review of every record by a senior research assistant or program manager; targeted review of high-value projects; and currency exchange and deflation calculation.  As a last step, the data are vetted by a large network of internal and external reviewers to identify potential errors or missing data.  

It is our hope that through the protocols and processes of the TUFF methodology, we can minimize errors, biases, and gaps and produce the most consistent, complete and replicable dataset possible. That being said, all open source data collection initiatives like this one are subject to limitations and biases (e.g., Muchapondwa et al. 2016). As such, the interactive platform available at china.aiddata.org is designed to facilitate the collection of feedback from a distributed network of users who may possess valuable information about relevant projects and financial transfers. Users who have knowledge of any additional projects or information to update any existing projects can “suggest a project”, “confirm”, “challenge”, or “comment” on any project on the china.aiddata.org platform. Once program managers at AidData receive these notifications, they seek to vet and verify the information, and when appropriate, they revise project records on the basis of credible information provided by external parties.