Back to History Current Version

Ed versus EdTech

Last registered on May 17, 2020

Pre-Trial

Trial Information

General Information

Title
Ed versus EdTech
RCT ID
AEARCTR-0003086
Initial registration date
June 14, 2018

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
June 18, 2018, 5:00 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
May 17, 2020, 1:49 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Freeman Spogli Institute for International Studies, Stanford University

Other Primary Investigator(s)

PI Affiliation
University of California at Santa Cruz
PI Affiliation
Stanford University
PI Affiliation
Stanford University

Additional Trial Information

Status
On going
Start date
2017-10-11
End date
2020-12-31
Secondary IDs
Abstract
Computer assisted learning (CAL), online courses, MOOCs, and other forms of educational technology have been touted as revolutionizing the way in which students are educated. Of particular importance is the question of whether technology-based interventions can improve education for disadvantaged populations. Although EdTech is being rapidly deployed throughout the developed and developing world there is limited evidence on whether and how it affects academic outcomes. Ultimately, it is important to determine if EdTech has net effects relative to other, more traditional forms of instruction and learning. The purpose of our study is to examine whether and why CAL has a positive impact on student achievement in general but also relative to more traditional forms of supplemental learning (such as pencil and paper workbooks). To fulfill this purpose, we conduct an RCT involving more than 4,000 fourth to sixth grade schoolchildren that are boarding in 130 schools in rural China. The RCT includes three treatment arms: a supplemental CAL arm, a traditional supplemental learning (pencil and paper workbooks) arm, and a control arm. The supplemental learning offered by the first two treatment arms is the same in terms of time and content. The traditional supplemental learning arm serves as an additional comparison group (in addition to the control arm) to the supplemental CAL arm. With this design, we can estimate the effect of supplemental learning (CAL or workbook) in addition to the net effect of CAL (versus traditional learning) on student outcomes. Thus, we can test whether it is the “Ed” or the “Tech” in “EdTech” that makes a difference.
External Link(s)

Registration Citation

Citation
Fairlie, Robert et al. 2020. "Ed versus EdTech." AEA RCT Registry. May 17. https://doi.org/10.1257/rct.3086-2.1
Former Citation
Fairlie, Robert et al. 2020. "Ed versus EdTech." AEA RCT Registry. May 17. https://www.socialscienceregistry.org/trials/3086/history/68583
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2017-11-01
Intervention End Date
2018-06-15

Primary Outcomes

Primary Outcomes (end points)
Student math achievement on 35 minute exam during endline survey
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Please see the pre-analysis plan document
Secondary Outcomes (explanation)
Please see the pre-analysis plan document

Experimental Design

Experimental Design
We designated each of 27 country-grades (9 counties and primary school grades 4-6) in our sample as strata or blocks. We then randomly allocated classes (one class per school-grade) within these strata to one of three different treatment conditions (T1 = Supplemental CAL, T2 = Supplemental Workbook, or C = Control):
A. Supplemental CAL (T1) 116 classes (in 88 schools)
B. Supplemental Workbook (T2) 118 classes (in 86 schools)
C. Control (C) 118 classes (in 85 schools)
Experimental Design Details
Randomization Method
Randomization done in office by a computer
Randomization Unit
Class
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
352 classes
Sample size: planned number of observations
Approximately 4,000 students
Sample size (or number of clusters) by treatment arms
A. Supplemental CAL (T1) 116 classes (in 88 schools)
B. Supplemental Workbook (T2) 118 classes (in 86 schools)
C. Control (C) 118 classes (in 85 schools)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Power calculations were conducted before the beginning of the trial using Optimal Design software (Spybrook et al., 2009). We conservatively used the following parameters to estimate the sample size for the study: • Intraclass correlation coefficient (adjusted for county-grade fixed effects): 0.10 • Average number of boarding students per class: 11 • R-squared of 0.40 With alpha = 0.05 and beta = 0.8, we estimated that we would need 115 classes per treatment arm for a minimum detectable effect size (MDES) of 0.14 SDs. We expect to only lose a small amount of statistical power due to (low rates of) student attrition (from the start to the end of the school year). Using information from past large-scale surveys in rural primary schools in western China, we assume an attrition rate of approximately 5%.
IRB

Institutional Review Boards (IRBs)

IRB Name
Stanford University
IRB Approval Date
2018-09-19
IRB Approval Number
35635
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials