Timor-Leste Data Uptake Evaluation

Last registered on March 17, 2017

View Trial History

Pre-Trial

Trial Information

General Information

Title

Timor-Leste Data Uptake Evaluation

RCT ID

AEARCTR-0001557

Initial registration date

August 31, 2016

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

August 31, 2016, 6:48 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated

March 17, 2017, 9:45 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country

Timor-Leste

Region

Southeast Asia

Primary Investigator

Name

Alena Stern

Affiliation

AidData

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Katherine Nolan

PI Affiliation

AidData

Contact Investigator

PI Name

S.P. Harish

PI Affiliation

New York University

Contact Investigator

PI Name

Ariel BenYishay

PI Affiliation

William and Mary / AidData

Contact Investigator

Additional Trial Information

Status

On going

Start date

2016-07-10

End date

2017-07-31

Keywords

Governance

Additional Keywords

Open Data

JEL code(s)

Secondary IDs

Abstract

This study proposes to evaluate the effectiveness of interventions to promote greater use of publicly available aid information in the Government of Timor-Leste's Aid Transparency Portal (ATP) in the decision-making of development policy makers and practitioners in Timor-Leste. The ATP is managed by the Development Partnership Management Unit (DPMU) in the Government of Timor-Leste Ministry of Finance (MoF). Under a five year cooperative agreement (October 2012-September 2017) funded by the USAID Global Development Lab under the Higher Education Solutions Network Program, AidData has worked with DPMU since August 2013 to geocode projects in the ATP by assigning precise latitude and longitude coordinates to project locations, enabling this information to be easily visualized and analyzed. Development practitioners in Timor-Leste can use this information to more precisely target aid programs to populations of need, coordinate their efforts with other actors, and evaluate the impact of their programs.

However, uptake of this information remains limited by government line ministry officials, donor partner personnel, and non-governmental organization staff in Timor-Leste. AidData conducted an assessment in February 2016 to identify challenges to data uptake and opportunities to increase use. The assessment revealed that three of the key barriers to use of the geocoded ATP data were 1) lack of awareness of the ATP, 2) limited capacity to analyze geocoded data for relevant use cases, and 3) poor internet connectivity to access ATP. Based on these findings, Development Gateway will implement the following interventions to promote greater use of the ATP data:

1. Outreach Video: A five-minute video that provides an overview of the features of the ATP and its potential uses. This video will be shown to participants following the baseline survey.
2. Training: A two-day training which will provide in-depth instruction on how to use the ATP portal to conduct key data analyses. This training will be provided to participants following the outreach video and prior to the midline survey.
3. Offline Data Access: ATP data and informational reports will be provided directly to participants via USB drives, circumventing the need to go online to access the data. This treatment will be provided between the midline and endline surveys.

The research team will evaluate the impact of these interventions on promoting increased use of ATP data among government line ministries, INGOs, local NGOs, donor partners, and implementing partners in Timor-Leste.

External Link(s)

Registration Citation

Citation

BenYishay, Ariel et al. 2017. "Timor-Leste Data Uptake Evaluation." AEA RCT Registry. March 17. https://doi.org/10.1257/rct.1557-2.0

Former Citation

BenYishay, Ariel et al. 2017. "Timor-Leste Data Uptake Evaluation." AEA RCT Registry. March 17. https://www.socialscienceregistry.org/trials/1557/history/15072

Sponsors & Partners

Experimental Details

Interventions

Intervention(s)

1.Outreach Video: A five-minute video that provides an overview of the features of the ATP and its potential uses. This video will be shown to participants following the baseline survey.
2.Training: A two-day training which will provide in-depth instruction on how to use the ATP portal to conduct key data analyses. This training will be provided to participants following the outreach video and prior to the midline survey.
3.Offline Data Access: ATP data and informational reports will be provided directly to participants via USB drives, circumventing the need to go online to access the data. This treatment will be provided between the midline and endline surveys.

Intervention Start Date

2016-08-08

Intervention End Date

2017-05-31

Primary Outcomes

Primary Outcomes (end points)

Perceived usefulness of the Aid Transparency Portal (ATP); Knowledge, familiarity, and trust of ATP; Use of ATP in decision-making; Aid allocation outcomes.

Primary Outcomes (explanation)

3. Key data sources and construction of variables:

There are four key data sources:
1. Baseline, Midline and Endline surveys of participants:
● Q951 I’m going to read you a list of options. Please tell me what applies to you. As part of your regular job functions, which sources of data and evidence do you use? These are sources that you personally use, not your organization. Please read all options word for word as listed, and select all that apply. Click on the source if the participant does use it.
● Q1183 In the table below, please indicate how much you trust each of the data sources listed. Please select one answer for every data source listed in the table:
● Q1184 In the table below, please select for which job responsibilities you use the following sources of data. Please select all of the job responsibilities that apply for each source of data. Please select at least one option for every data source listed:
● Q971 Why do you choose the Aid Transparency Portal data source in your work? Please choose up to the top 3 reasons. Enumerator: Read all options. Choose the top three. You can turn the tablet around to show them the list if that will help them choose.
● Q1027 How often do you use the ATP in your work?
● Q1026 How useful is the ATP for your work? Read all options
● Q1017 How useful is this information source? Based on a scale from 0 to 10. 0 being extremely useless. 10 being extremely useful.
● Q1018 How do you use this information? Please read all the options.
2. Post-Video Questions
● Q1317 How useful would the ATP for your work?
● Q1318 How would you use the ATP in your work? Read all options
3. Outcomes of the Lab in the Field (LITF) Activity at Baseline and Midline
● Q1320: LITF aid allocation decision
● Q1321 Please justify your decision. (The allocation decision)
● Q1322 What information sources did you use to make your decision?
● Q1324 Why did you choose ATP Data?
● Q1335 If you were going to make a recommendation to your boss about where your next project will be located, what would that recommendation be?
● Q887 How did you decide on this recommendation?
4. Post-Training Questions
● Q6: Please tell us if you agree or disagree with the following statements by circling the option that best represents your opinion: I will use the ATP in my work
● Q7: Please tell us if you agree or disagree with the following statements by circling the option that best represents your opinion: This course was not beneficial for my work
● Q8: Please tell us if you agree or disagree with the following statements by circling the option that best represents your opinion: This training will help me be more efficient at work
● Q9: Please tell us if you agree or disagree with the following statements by circling the option that best represents your opinion: The ATP is a valuable resource for my work
5. Outcomes of Government Data Request (note that access to this data is tentative pending Government of Timor Leste approval).

Secondary Outcomes

Secondary Outcomes (end points)

Secondary Outcomes (explanation)

Experimental Design

The target audience for these interventions are individuals that participate in decisions regarding the design, coordination, and implementation of development projects within government line ministries, INGOs, local NGOs, donor partners, and implementing partners in Timor-Leste. We will randomly select 200 participants from these target audiences to comprise our study sample. For comparability within the sample and feasibility of treatment implementation, all participants selected will be based in Dili, Timor-Leste. Each of the treatments will be randomly assigned across this sample with 50% of the sample receiving each treatment and 50% in the control group. We will randomly assign the outreach video and training treatments to 50% of our sample. Then we will randomly assign 50% of the earlier treatment group and 50% of the earlier control group to the offline data access treatment. The first randomization will be stratified by organization type and seniority and the second randomization will be stratified by organization type, seniority, and receipt of the outreach video/training treatments.

Experimental Design Details

Randomization Method

Randomization performed in office using STATA.

Randomization Unit

Individual

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

200 government line ministry, INGO, local NGO, donor partner, and implementing partner personnel.

Sample size: planned number of observations

200 government line ministry, INGO, local NGO, donor partner, and implementing partner personnel.

Sample size (or number of clusters) by treatment arms

50 individuals control
50 individuals outreach video/ training treatment
50 individuals offline data treatment
50 individuals outreach video/ training treatment and offline data treatment

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Supporting Documents and Materials

IRB