x

Please fill out this short user survey of only 3 questions in order to help us improve the site. We appreciate your feedback!
On Adolescent Human Capital Production: Evidence from a Structurally-Motivated Field Experiment Across Three School Districts
Last registered on October 15, 2020

Pre-Trial

Trial Information
General Information
Title
On Adolescent Human Capital Production: Evidence from a Structurally-Motivated Field Experiment Across Three School Districts
RCT ID
AEARCTR-0006530
Initial registration date
October 14, 2020
Last updated
October 15, 2020 12:41 PM EDT
Location(s)
Primary Investigator
Affiliation
University of Chicago; Australian National University
Other Primary Investigator(s)
PI Affiliation
Brigham Young University
PI Affiliation
Queen's University
PI Affiliation
Washington University in St Louis
PI Affiliation
University of Chicago
Additional Trial Information
Status
Completed
Start date
2014-05-01
End date
2014-05-22
Secondary IDs
Abstract
We leverage a field experiment across three distinct school districts to identify key pieces of a structural model of adolescent human capital production. Our focus is inspired by the contemporary psychology of education literature, which expresses learning as a function of the ratio of the time spent on learning to the time needed to learn. By capturing two crucial student-level unobservables---which we denote as academic efficiency (turning inputs into outputs) and time preference (motivation)---our field experiment lends insights into the underpinnings of adolescent skill formation and provides a novel view of how to lessen racial and gender achievement gaps.
External Link(s)
Registration Citation
Citation
Cotton, Chris et al. 2020. "On Adolescent Human Capital Production: Evidence from a Structurally-Motivated Field Experiment Across Three School Districts." AEA RCT Registry. October 15. https://doi.org/10.1257/rct.6530-1.0.
Experimental Details
Interventions
Intervention(s)
Our research design builds on our study-choice and skill-formation framework to bring together experimental and structural methods to quantify unobserved student characteristics. Our strategy uses the student choice model as a basis for an econometric framework, where field experimental methods shape a data-generating process with the requisite sets of observables and variation to enable identification of the structural parameters at the individual level. This data-generating process is also carefully crafted to be as true to students' everyday academic choices and experiences as possible. With this in mind, we conducted the field experiment among fifth- and sixth-grade students in three Illinois school districts. We offered varying monetary incentives for completion of extra-curricular learning activities on a math study website that we developed.

Intervention Start Date
2014-05-12
Intervention End Date
2014-05-21
Primary Outcomes
Primary Outcomes (end points)
Number of Math Quizzes solved, Time Taken to Solve Quizzes and Estimating Structural parameters.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
We worked closely with fifth- and sixth-grade math teachers across the three participating school districts to implement the field experiment. The major research advantage to this partnership was that participation in the study was on an opt-out basis, allowing the research team to achieve a sample that was much more representative of the local populations our partner schools serve. The primary feature of the experiment was a website on which the students could complete up to 80 mathematics modules, referred to as quizzes, across five general topics. Students had access to the website for 10 days and could complete as many of the quizzes as they chose. Throughout the process, our web server monitored students' activities and tallied successful completion of quizzes. Piece-rate incentives were offered for task completion on the website, based on the number of quizzes completed successfully, rather than on time spent. We also measured proficiency using in-classroom mathematics assessments.

Our experimental design centers on randomized incentives. We adopted a linear piece-rate schedule with a constant marginal piece-rate that would be easy for adolescents to understand. We varied both the base payment, for showing up and completing the minimum amount of work, and the marginal piece-rate payment. No payments are offered until a child has passed 2 quizzes, which ensures a within-student panel for each individual. Each student was randomly assigned to one of three possible contract groups: (base pay: $15, bonus: $0.75), (base pay: $10, bonus: $1.00), and (base pay: $5, bonus: $1.25). Assignment was at the individual level, resulting in treatment variation within school, grades, and classrooms.

Our Experiment Timeline:

In summary, the experiment took place as follows.

1) Students took a pre-test and survey administered by their teachers in class.
2) Students were randomly assigned a wage contract, and provided with information about the experiment, including the website and their earnings potential.
3)For the next 10 days, student work on the website counted towards their compensation. Following the 10 day period, they were paid based on the number of quizzes successfully completed during that time.
4) Student took a post-test and a second survey administered by their teachers in class.


Experimental Design Details
Randomization Method
Assignment was at the individual level, resulting in treatment variation within school, grades, and classrooms.

More specifically, our randomization algorithm first separated students into race-gender-school-grade bins. Within each bin it balanced on pre-test scores by ordering students according to their score and randomly assigning consecutive blocks of 3 similar-score students to contract groups 1, 2, and 3. The algorithm then repeated this process thousands of times, and selected the random assignment that minimized overall correlations between treatment status and balance variables.

Our pre-exam materials were produced and organized in such a way that they could be collected from teachers and rapidly processed so as to allow for balancing on initial math proficiency during randomization. Exams were administered to students toward the end of the school week, and they were processed, randomization executed, and personalized instruction materials for each student were produced over the weekend for in-classroom delivery by math teachers the following Monday. Each student participant received a personalized letter in a sealed envelope, containing login credentials, instructions for accessing the website, and their individual piece-rate incentive contract. They were also promised prompt delivery of payments within 2 weeks following the end of the experiment.
Randomization Unit
individual level
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
1676 pupils.
Sample size: planned number of observations
1676 pupils.
Sample size (or number of clusters) by treatment arms
contract with base pay 15, and bonus .75 has 557 pupils; contract with base 10 and bonus 1 has 559 pupils; contract with base 5 and bonus 1.25 has 560 pupils.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
IRB Approval Date
IRB Approval Number
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports, Papers & Other Materials
Relevant Paper(s)
REPORTS & OTHER MATERIALS