About TIME (Technology in Math Education): Experiments 1 & 2

Last registered on May 05, 2022

Pre-Trial

Trial Information

General Information

Title
About TIME (Technology in Math Education): Experiments 1 & 2
RCT ID
AEARCTR-0009345
Initial registration date
May 05, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 05, 2022, 12:57 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Primary Investigator

Affiliation
University of Chicago

Other Primary Investigator(s)

PI Affiliation
University of Chicago
PI Affiliation
University of Chicago

Additional Trial Information

Status
On going
Start date
2022-04-18
End date
2022-09-30
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
About TIME (Technology in Math Education) explores how technology can best be used to promote the acquisition of math skills among young children and reduce the gap in skill development between high- and low-income children. We hypothesize that digital apps may improve young children’s math skills over the long run compared to similar non-digital learning materials both because children are likely to spend more time using digital apps compared to non-digital learning materials and because digital apps may be more efficient at conveying math skill. About TIME will proceed in two parts. The first is a series of field experiments to help us understand how technology works to promote learning (corresponding to this registration). The second is a large-scale randomized controlled trial intended to compare the increase in math skills among preschool-age children who are provided with a tablet containing high-quality math apps and a group of children provided with high-quality non-digital materials of the sort recommended by experts in math learning and intended to promote the same skills as the digital apps. The results of the About TIME Project will: 1. compare the increase in math skills for children provided with a digital tablet containing high-quality math apps to the increase for children provided with high-quality non-digital materials intended to convey the same skills, 2. estimate the extent to which parent characteristics and barriers to parent engagement influence the effect of both digital and non-digital learning tools in building child math skill, 3. estimate how parent involvement moderates the effect of digital apps relative to non-digital materials on children’s math skills, 4. estimate how digital apps affect children’s and parents’ enthusiasm for math, 5. describe which apps are used the most, when are they used, and which features of apps are most often used by children.


External Link(s)

Registration Citation

Citation
Kalil, Ariel, Haoxuan Liu and Susan Mayer. 2022. "About TIME (Technology in Math Education): Experiments 1 & 2." AEA RCT Registry. May 05. https://doi.org/10.1257/rct.9345-1.0
Experimental Details

Interventions

Intervention(s)
Experiment 1 (4 treatment arms):
Parents will be told that the goal of the session is for children to learn a specific skill, for example matching number symbols to quantities or naming large number symbols. Parent-child pairs will be randomly assigned to receive apps or non-digital learning materials. Parents will then be randomly assigned within the two modalities, either to be present or absent for the experiment. Holding the time constant, any difference in skill acquisition between the beginning and end of the experiment will be due to differences in the efficiency of the materials in the two scenarios where the parent is either present or absent.

Experiment 2 (2 treatment arms):
In this experiment, we randomly assign parents to a treatment in which we temporarily increase their cognitive load or to a treatment in which we do not increase cognitive load. There are standard methodologies for temporarily increasing cognitive load, such as asking participants to count backward by 7s from 100 and other similar tasks that tax cognition. As in experiment 1, we test how much children learn from digital or non-digital materials. Because low-income parents are more likely to have a high cognitive load, this can also potentially explain why low-income parents engage less with their children in math activities.
Intervention (Hidden)
Intervention Start Date
2022-04-18
Intervention End Date
2022-09-30

Primary Outcomes

Primary Outcomes (end points)
The efficiency of digital v. non-digital math learning materials in conveying numeracy skill to preschool-aged children analyzed by parent socio-economic status; Role of parent engagement in conveying math skill with digital v. non-digital math learning materials. Both outcomes hold time, discreet skill being conveyed, and pedagogy constant

Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Qualitative analysis of parent-child interactions; math enthusiasm
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The intent of the field experiments is to test several potential explanations for how digital apps and non-digital materials influence math learning. There are two main experiments. Experiment 1 asks whether digital apps are more efficient at conveying math skills and what role the parent plays in skill development. Experiment 2 asks how parents’ cognitive load affects children’s learning with digital and non-digital materials.

The experiments have the same general format in that parent-child pairs will come to the experimental site, consent to the experiment, be randomized to a treatment group or a control group, and participate in a 30–45-minute experiment. Children will be given a brief assessment of a specific math skill at the beginning and end of the experiment and the difference in score will be the main outcome for each experimental comparison. Each experiment will also assess children’s enthusiasm for learning, measured by whether when given the chance children choose to spend more time on another math activity. All experiments will be live coded by researchers and video recorded for auditing purposes.

We will survey parents at baseline and at the end of the intervention to collect data on family demographic characteristics, parents’ reports of the home learning environment, parents’ beliefs and expectations about their children’s math skill development, and parents’ expected return on their time and material investments in children’s math skill development. We will also ask questions that measure parents’ math anxiety and growth mindset and parents’ reports of the structural, behavioral, and cognitive barriers that stand in the way of their child’s math skill development. These will be used as covariates in the analysis of changes in assessment scores. We will manage the equal distribution of low vs. high income into those groups by stratifying on income within the randomization procedure.
Experimental Design Details
Randomization Method
Parent-child dyads recruited across 15 institutional sites were randomly assigned to a treatment condition immediately upon recruitment.
Randomization Unit
Our unit of randomization is parent-child dyads
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
We don't design clusters, so the planned number of clusters is 900 parent-child dyads.
Sample size: planned number of observations
900 parent-child dyads
Sample size (or number of clusters) by treatment arms
75 low/75 high-income parent-child dyads in each of the six treatment arms mentioned in the Intervention section
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Our minimum detectable effect size for the main Intention-to-Treat effect is designed to be 0.27 at the .05 level of statistical significance and with an 80 percent chance of deriving statistically significant impacts assuming a two-sided test of the null hypothesis.
IRB

Institutional Review Boards (IRBs)

IRB Name
The University of Chicago Institutional Review Board
IRB Approval Date
2022-01-19
IRB Approval Number
IRB21-2002

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials