Take-Up, Use, and Effectiveness of Remote Learning Technologies

We have two information treatments and two treatments that reduce the price of educational inputs.

The information treatments are:
- Treatment 1 – Remote learning information: Information and reminders about government remote learning initiatives. We split the information salience treatment into two sub-treatments:
o 1A – “TV”: SMS information/reminders about TV lessons
o 1B – “TV+Internet”: SMS information/reminders about TV lessons and internet lessons
- Treatment 2 – Adaptive learning: Information about an adaptive learning internet resource

The cost reduction treatments are:
- Treatment 3 – Data subsidy: Reduce costs of using internet remote learning technologies by providing a free 1-month data package for 10 GB.
- Treatment 4 – Teacher support: Pair students with a teacher who provides weekly check-ins and support.

2021-02-01

2021-04-18

• Parental behaviors:
o Time and economic investments in children’s education
• Take-up and usage of learning resources
• Student time investment
• Student learning (performance on math and Bangla questions)
• Student engagement and re-enrollment

• Parental beliefs and expectations on re-enrollment and grade completion
• Child expectations on re-enrollment and grade completion
• Student educational aspirations and expectations
• Student outside options

Our sample consists of households that have (a) at least one child in grades 6-10 (grades 7-11 in January 2021) and (b) have at least one smartphone in the house. The entire sample was cross-randomized in evenly sized cells across Treatment 1 (information and reminders about remote TV lessons and online platforms) and Treatment 2 (adaptive learning).

Treatment arm 3 (data subsidy) is cross randomized equally across Treatments 2 and/or 1b (TV + internet). Treatment arm 4 (teacher support) was initially cross-randomized to 1/4 of students (teacher support) across Treatments 1a and 1b (TV only and TV + internet only), though we increased the share assigned to treatment in response to substantial treatment attrition.

Our main empirical specifications will estimate intention-to-treat effects, reflecting the causal impact of assignment to each treatment arm on our outcomes of interest.

We recruited households based on numbers obtained from three sources: A sample of numbers obtained through Random Digit Dialing (RDD), a list of numbers of parents of recipients of the government Secondary School Stipend Program, and a list of phone numbers of students who had previously enrolled in the Konnect webpage. The RDD sample is aimed to be broadly representative of all smartphone users in Bangladesh and will require more screening to select eligible households. The second sample (recipients of the Secondary School Stipend) will include more underprivileged households and it is targeted to the age range of interest. The third sample (students enrolled in the Konnect webpage) will likely have slightly higher socioeconomic status (SES) than average, but likely higher smartphone penetration. It includes those households for whom we already know whether they have children eligible for our study (though in higher grades on average).

We stratified by the following factors:
• Household income (5 categories).
• Sample source: whether sample was drawn from Konnect, Secondary School Stipend or RDD databases (3 categories).
• Child gender: Whether households had only male, only female, or both male and female children in grades 6-10 (3 categories).
• Internet: Whether the household had access to at least one smartphone with an active internet connection (2 categories).

Our main specification will pool smaller treatment arms, specifically the TV and TV + internet arms, for which we do not expect substantial differences in treatment impacts. However, we will disaggregate these treatment arms for the subset of hypotheses about any differences in the effects of information about the TV program only vs. TV and internet resources.

Our main outcomes are estimated at the child level, while treatment assignment is randomized at the household level. Because some households have more than one child in grades 6-10, we estimate our models at the child level and cluster our standard errors at the household level to reflect the level of randomization (Abadie et al., 2017).

We follow Jones et al. (2019) to select covariates. The first set will include our set of stratification cell fixed effects only, which effectively control for those variables. The second set will those fixed effects along with include baseline variables that are good predictors of the outcome variable. We will estimate a LASSO regression with five-fold cross validation, including all variables from our baseline data. We will present a table that shows how our results compare across the two specifications, and we will choose one as our preferred specification going forward.

Randomization done in office by a computer

Households

No

None

6,746 households

Equal size treatment arms across the main two groups:

- No outreach, no encouragement (control): 22,500
- Encouragement, no outreach to teachers: 22,500
- Outreach to teachers, no encouragement: 22,500
- Outreach to teachers, encouragement: 22,500

Within the encouragement treatment arm (45,000 treated), we will randomly assign students to the three types of information if their household has access to a smartphone and cable television. We do not know in advance what this sample size will be, but the treatment assignment will be distributed evenly (1/3, 1/3, 1/3) for that group. If 80% of households have access to a smartphone and cable television (36,000), then the sample size for treatment arms 2a, 2b, and 2c will be 12,000 each.

Data package reduction: The exact sample size of the data reduction package treatment arm will depend on budgetary constraints, but the randomization strategy is the following:
- Take the sample of 45,000 assigned to encouragement treatment
- Restrict to those who have access to a smart phone and are subscribers of our telecom partner
- Exclude treatment 2a (information about TV only)
- Among those, randomly select X to receive the data package subsidy, and the remainder will not receive the subsidy.

Most outcomes will be collected by IVR for the full sample. Outcomes requiring in-person phone survey (such as expectations, aspirations, and mental health outcomes) will be only collected for a subsample of individuals (around 10,000).