Take-Up, Use, and Effectiveness of Remote Learning Technologies

Last registered on April 30, 2021

Pre-Trial

Trial Information

General Information

Title
Take-Up, Use, and Effectiveness of Remote Learning Technologies
RCT ID
AEARCTR-0006191
Initial registration date
August 11, 2020
Last updated
April 30, 2021, 3:58 PM EDT

Locations

Region

Primary Investigator

Affiliation
Stockholm University IIES

Other Primary Investigator(s)

PI Affiliation
U. Wisconsin-Madison
PI Affiliation
University of Vermont

Additional Trial Information

Status
On going
Start date
2020-11-01
End date
2021-07-15
Secondary IDs
Abstract
Governments and educational organizations worldwide are trying to quickly adapt to the unprecedented circumstances created by the pandemic, by developing or scaling up distance education modalities to continue delivering educational content to students and maintain students’ connection to formal education. This study evaluates four interventions designed to reduce the barriers to remote education. The first two treatment arms target information constraints: The first provides information about remote learning options, and the second provides information about an adaptive learning internet resource. The last two treatment arms target price constraints: The third reduces the cost of internet learning activities by providing a discounted data package, and the fourth pairs students with a teacher that provides weekly support.

The main outcomes of interest include parents' and students’ time and economic investments in remote education, take up and usage of remote learning resources, and learning outcomes.
External Link(s)

Registration Citation

Citation
Beam, Emily, Priya Mukherjee and Laia Navarro-Sola. 2021. "Take-Up, Use, and Effectiveness of Remote Learning Technologies." AEA RCT Registry. April 30. https://doi.org/10.1257/rct.6191-2.0
Sponsors & Partners

Sponsors

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details

Interventions

Intervention(s)
We have two information treatments and two treatments that reduce the price of educational inputs.

The information treatments are:
- Treatment 1 – Remote learning information: Information and reminders about government remote learning initiatives. We split the information salience treatment into two sub-treatments:
o 1A – “TV”: SMS information/reminders about TV lessons
o 1B – “TV+Internet”: SMS information/reminders about TV lessons and internet lessons
- Treatment 2 – Adaptive learning: Information about an adaptive learning internet resource

The cost reduction treatments are:
- Treatment 3 – Data subsidy: Reduce costs of using internet remote learning technologies by providing a free 1-month data package for 10 GB.
- Treatment 4 – Teacher support: Pair students with a teacher who provides weekly check-ins and support.
Intervention Start Date
2021-02-01
Intervention End Date
2021-04-18

Primary Outcomes

Primary Outcomes (end points)
Parental behaviors (Time and economic investments in children’s education), take-up and usage of learning resources, student time investment, student learning (performance on math and Bangla questions), student engagement and re-enrollment
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Parental beliefs and expectations on re-enrollment and grade completion, student educational aspirations and expectations, student outside options
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Our baseline sample consists of 7,576 households that have (a) at least one child in grades 6-10 (grades 7-11 in January 2021) and (b) have at least one smartphone in the house. The entire sample was cross-randomized in evenly sized cells across Treatment 1 (information and reminders about remote TV lessons and online platforms) and Treatment 2 (adaptive learning). 6,746 households agreed to be recontacted and form the sample for our midline and endline surveys.

Treatment arm 3 (data subsidy) is cross randomized equally across Treatments 2 and/or 1b (TV + internet). Treatment arm 4 (teacher support) was initially cross-randomized to 1/4 of students (teacher support) across Treatments 1a and 1b (TV only and TV + internet only), though we increased the share assigned to treatment in response to substantial treatment attrition.

Our main empirical specifications will estimate intention-to-treat effects, reflecting the causal impact of assignment to each treatment arm on our outcomes of interest.
Experimental Design Details
We recruited households based on numbers obtained from three sources: A sample of numbers obtained through Random Digit Dialing (RDD), a list of numbers of parents of recipients of the government Secondary School Stipend Program, and a list of phone numbers of students who had previously enrolled in the Konnect webpage. The RDD sample is aimed to be broadly representative of all smartphone users in Bangladesh and will require more screening to select eligible households. The second sample (recipients of the Secondary School Stipend) will include more underprivileged households and it is targeted to the age range of interest. The third sample (students enrolled in the Konnect webpage) will likely have slightly higher socioeconomic status (SES) than average, but likely higher smartphone penetration. It includes those households for whom we already know whether they have children eligible for our study (though in higher grades on average).


We stratified by the following factors:
• Household income (5 categories).
• Sample source: whether sample was drawn from Konnect, Secondary School Stipend or RDD databases (3 categories).
• Child gender: Whether households had only male, only female, or both male and female children in grades 6-10 (3 categories).
• Internet: Whether the household had access to at least one smartphone with an active internet connection (2 categories).


Our main specification will pool smaller treatment arms, specifically the TV and TV + internet arms, for which we do not expect substantial differences in treatment impacts. However, we will disaggregate these treatment arms for the subset of hypotheses about any differences in the effects of information about the TV program only vs. TV and internet resources.

Our main outcomes are estimated at the child level, while treatment assignment is randomized at the household level. Because some households have more than one child in grades 6-10, we estimate our models at the child level and cluster our standard errors at the household level to reflect the level of randomization (Abadie et al., 2017).

We follow Jones et al. (2019) to select covariates. The first set will include our set of stratification cell fixed effects only, which effectively control for those variables. The second set will those fixed effects along with include baseline variables that are good predictors of the outcome variable. We will estimate a LASSO regression with five-fold cross validation, including all variables from our baseline data. We will present a table that shows how our results compare across the two specifications, and we will choose one as our preferred specification going forward.
Randomization Method
Randomization done in office by a computer
Randomization Unit
Households
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
None
Sample size: planned number of observations
6,746 households
Sample size (or number of clusters) by treatment arms
Below, we provide an exhaustive list of treatment arms that arise from cross-randomizing each of the main treatment arms.

• 0. Control : 1,894
• 1a. TV : 525
• 1b. TV + internet : 253
• 2a. Adaptive learning : 947
• 2b. Adaptive learning + TV : 471
• 2c. Adaptive learning + TV + internet : 473
• 3a. Data subsidy + TV + internet : 288
• 3b. Data subsidy + Adaptive learning : 947
• 3c. Data subsidy + Adaptive learning + TV : 476
• 3d. Data subsidy + Adaptive learning + TV + internet : 474
• 4a. Teacher support + TV : 422
• 4b. Teacher support + TV + internet : 223
• 4c. Teacher support + Data subsidy + TV + internet : 183
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Innovations for Poverty Action Institutional Review Board #0004745
IRB Approval Date
2020-07-25
IRB Approval Number
15594
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials