x

NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
Timor-Leste Data Uptake Evaluation
Last registered on March 17, 2017

Pre-Trial

Trial Information
General Information
Title
Timor-Leste Data Uptake Evaluation
RCT ID
AEARCTR-0001557
Initial registration date
August 31, 2016
Last updated
March 17, 2017 9:45 AM EDT
Location(s)
Primary Investigator
Affiliation
AidData
Other Primary Investigator(s)
PI Affiliation
AidData
PI Affiliation
New York University
PI Affiliation
William and Mary / AidData
Additional Trial Information
Status
On going
Start date
2016-07-10
End date
2017-07-31
Secondary IDs
Abstract
This study proposes to evaluate the effectiveness of interventions to promote greater use of publicly available aid information in the Government of Timor-Leste's Aid Transparency Portal (ATP) in the decision-making of development policy makers and practitioners in Timor-Leste. The ATP is managed by the Development Partnership Management Unit (DPMU) in the Government of Timor-Leste Ministry of Finance (MoF). Under a five year cooperative agreement (October 2012-September 2017) funded by the USAID Global Development Lab under the Higher Education Solutions Network Program, AidData has worked with DPMU since August 2013 to geocode projects in the ATP by assigning precise latitude and longitude coordinates to project locations, enabling this information to be easily visualized and analyzed. Development practitioners in Timor-Leste can use this information to more precisely target aid programs to populations of need, coordinate their efforts with other actors, and evaluate the impact of their programs.

However, uptake of this information remains limited by government line ministry officials, donor partner personnel, and non-governmental organization staff in Timor-Leste. AidData conducted an assessment in February 2016 to identify challenges to data uptake and opportunities to increase use. The assessment revealed that three of the key barriers to use of the geocoded ATP data were 1) lack of awareness of the ATP, 2) limited capacity to analyze geocoded data for relevant use cases, and 3) poor internet connectivity to access ATP. Based on these findings, Development Gateway will implement the following interventions to promote greater use of the ATP data:

1. Outreach Video: A five-minute video that provides an overview of the features of the ATP and its potential uses. This video will be shown to participants following the baseline survey.
2. Training: A two-day training which will provide in-depth instruction on how to use the ATP portal to conduct key data analyses. This training will be provided to participants following the outreach video and prior to the midline survey.
3. Offline Data Access: ATP data and informational reports will be provided directly to participants via USB drives, circumventing the need to go online to access the data. This treatment will be provided between the midline and endline surveys.

The research team will evaluate the impact of these interventions on promoting increased use of ATP data among government line ministries, INGOs, local NGOs, donor partners, and implementing partners in Timor-Leste.
External Link(s)
Registration Citation
Citation
BenYishay, Ariel et al. 2017. "Timor-Leste Data Uptake Evaluation." AEA RCT Registry. March 17. https://doi.org/10.1257/rct.1557-2.0.
Former Citation
BenYishay, Ariel et al. 2017. "Timor-Leste Data Uptake Evaluation." AEA RCT Registry. March 17. http://www.socialscienceregistry.org/trials/1557/history/15072.
Experimental Details
Interventions
Intervention(s)
1.Outreach Video: A five-minute video that provides an overview of the features of the ATP and its potential uses. This video will be shown to participants following the baseline survey.
2.Training: A two-day training which will provide in-depth instruction on how to use the ATP portal to conduct key data analyses. This training will be provided to participants following the outreach video and prior to the midline survey.
3.Offline Data Access: ATP data and informational reports will be provided directly to participants via USB drives, circumventing the need to go online to access the data. This treatment will be provided between the midline and endline surveys.
Intervention Start Date
2016-08-08
Intervention End Date
2017-05-31
Primary Outcomes
Primary Outcomes (end points)
Perceived usefulness of the Aid Transparency Portal (ATP); Knowledge, familiarity, and trust of ATP; Use of ATP in decision-making; Aid allocation outcomes.
Primary Outcomes (explanation)
3. Key data sources and construction of variables:

There are four key data sources:
1. Baseline, Midline and Endline surveys of participants:
● Q951 I’m going to read you a list of options. Please tell me what applies to you. As part of your regular job functions, which sources of data and evidence do you use? These are sources that you personally use, not your organization. Please read all options word for word as listed, and select all that apply. Click on the source if the participant does use it.
● Q1183 In the table below, please indicate how much you trust each of the data sources listed. Please select one answer for every data source listed in the table:
● Q1184 In the table below, please select for which job responsibilities you use the following sources of data. Please select all of the job responsibilities that apply for each source of data. Please select at least one option for every data source listed:
● Q971 Why do you choose the Aid Transparency Portal data source in your work? Please choose up to the top 3 reasons. Enumerator: Read all options. Choose the top three. You can turn the tablet around to show them the list if that will help them choose.
● Q1027 How often do you use the ATP in your work?
● Q1026 How useful is the ATP for your work? Read all options
● Q1017 How useful is this information source? Based on a scale from 0 to 10. 0 being extremely useless. 10 being extremely useful.
● Q1018 How do you use this information? Please read all the options.
2. Post-Video Questions
● Q1317 How useful would the ATP for your work?
● Q1318 How would you use the ATP in your work? Read all options
3. Outcomes of the Lab in the Field (LITF) Activity at Baseline and Midline
● Q1320: LITF aid allocation decision
● Q1321 Please justify your decision. (The allocation decision)
● Q1322 What information sources did you use to make your decision?
● Q1324 Why did you choose ATP Data?
● Q1335 If you were going to make a recommendation to your boss about where your next project will be located, what would that recommendation be?
● Q887 How did you decide on this recommendation?
4. Post-Training Questions
● Q6: Please tell us if you agree or disagree with the following statements by circling the option that best represents your opinion: I will use the ATP in my work
● Q7: Please tell us if you agree or disagree with the following statements by circling the option that best represents your opinion: This course was not beneficial for my work
● Q8: Please tell us if you agree or disagree with the following statements by circling the option that best represents your opinion: This training will help me be more efficient at work
● Q9: Please tell us if you agree or disagree with the following statements by circling the option that best represents your opinion: The ATP is a valuable resource for my work
5. Outcomes of Government Data Request (note that access to this data is tentative pending Government of Timor Leste approval).
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
The target audience for these interventions are individuals that participate in decisions regarding the design, coordination, and implementation of development projects within government line ministries, INGOs, local NGOs, donor partners, and implementing partners in Timor-Leste. We will randomly select 200 participants from these target audiences to comprise our study sample. For comparability within the sample and feasibility of treatment implementation, all participants selected will be based in Dili, Timor-Leste. Each of the treatments will be randomly assigned across this sample with 50% of the sample receiving each treatment and 50% in the control group. We will randomly assign the outreach video and training treatments to 50% of our sample. Then we will randomly assign 50% of the earlier treatment group and 50% of the earlier control group to the offline data access treatment. The first randomization will be stratified by organization type and seniority and the second randomization will be stratified by organization type, seniority, and receipt of the outreach video/training treatments.
Experimental Design Details
Randomization Method
Randomization performed in office using STATA.
Randomization Unit
Individual
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
200 government line ministry, INGO, local NGO, donor partner, and implementing partner personnel.
Sample size: planned number of observations
200 government line ministry, INGO, local NGO, donor partner, and implementing partner personnel.
Sample size (or number of clusters) by treatment arms
50 individuals control
50 individuals outreach video/ training treatment
50 individuals offline data treatment
50 individuals outreach video/ training treatment and offline data treatment
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
WILLIAM AND MARY PROTECTION OF HUMAN SUBJECTS COMMITTEE
IRB Approval Date
2016-06-06
IRB Approval Number
PHSC-2016-05-23-11245-amstern01
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports, Papers & Other Materials
Relevant Paper(s)
REPORTS & OTHER MATERIALS