About TIME (Technology in Math Education): Experiments 1 & 2

Last registered on May 05, 2022

View Trial History

Pre-Trial

Trial Information

General Information

Title

About TIME (Technology in Math Education): Experiments 1 & 2

RCT ID

AEARCTR-0009345

Initial registration date

May 05, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

May 05, 2022, 12:57 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country

United States of America

Region

Illinois

Primary Investigator

Name

Haoxuan Liu

Affiliation

University of Chicago

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Ariel Kalil

PI Affiliation

University of Chicago

Contact Investigator

PI Name

Susan Mayer

PI Affiliation

University of Chicago

Contact Investigator

Additional Trial Information

Status

On going

Start date

2022-04-18

End date

2022-09-30

Keywords

Behavior, Education, Other

Additional Keywords

Parent engagement, family engagement, poverty and inequality, math, early childhood, preschool

JEL code(s)

Secondary IDs

Prior work

This trial does not extend or rely on any prior RCTs.

Abstract

About TIME (Technology in Math Education) explores how technology can best be used to promote the acquisition of math skills among young children and reduce the gap in skill development between high- and low-income children. We hypothesize that digital apps may improve young children’s math skills over the long run compared to similar non-digital learning materials both because children are likely to spend more time using digital apps compared to non-digital learning materials and because digital apps may be more efficient at conveying math skill. About TIME will proceed in two parts. The first is a series of field experiments to help us understand how technology works to promote learning (corresponding to this registration). The second is a large-scale randomized controlled trial intended to compare the increase in math skills among preschool-age children who are provided with a tablet containing high-quality math apps and a group of children provided with high-quality non-digital materials of the sort recommended by experts in math learning and intended to promote the same skills as the digital apps. The results of the About TIME Project will: 1. compare the increase in math skills for children provided with a digital tablet containing high-quality math apps to the increase for children provided with high-quality non-digital materials intended to convey the same skills, 2. estimate the extent to which parent characteristics and barriers to parent engagement influence the effect of both digital and non-digital learning tools in building child math skill, 3. estimate how parent involvement moderates the effect of digital apps relative to non-digital materials on children’s math skills, 4. estimate how digital apps affect children’s and parents’ enthusiasm for math, 5. describe which apps are used the most, when are they used, and which features of apps are most often used by children.

External Link(s)

Registration Citation

Citation

Kalil, Ariel, Haoxuan Liu and Susan Mayer. 2022. "About TIME (Technology in Math Education): Experiments 1 & 2." AEA RCT Registry. May 05. https://doi.org/10.1257/rct.9345-1.0

Sponsors & Partners

Experimental Details

Interventions

Intervention(s)

Experiment 1 (4 treatment arms):
Parents will be told that the goal of the session is for children to learn a specific skill, for example matching number symbols to quantities or naming large number symbols. Parent-child pairs will be randomly assigned to receive apps or non-digital learning materials. Parents will then be randomly assigned within the two modalities, either to be present or absent for the experiment. Holding the time constant, any difference in skill acquisition between the beginning and end of the experiment will be due to differences in the efficiency of the materials in the two scenarios where the parent is either present or absent.

Experiment 2 (2 treatment arms):
In this experiment, we randomly assign parents to a treatment in which we temporarily increase their cognitive load or to a treatment in which we do not increase cognitive load. There are standard methodologies for temporarily increasing cognitive load, such as asking participants to count backward by 7s from 100 and other similar tasks that tax cognition. As in experiment 1, we test how much children learn from digital or non-digital materials. Because low-income parents are more likely to have a high cognitive load, this can also potentially explain why low-income parents engage less with their children in math activities.

Intervention Start Date

2022-04-18

Intervention End Date

2022-09-30

Primary Outcomes

Primary Outcomes (end points)

The efficiency of digital v. non-digital math learning materials in conveying numeracy skill to preschool-aged children analyzed by parent socio-economic status; Role of parent engagement in conveying math skill with digital v. non-digital math learning materials. Both outcomes hold time, discreet skill being conveyed, and pedagogy constant

Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)

Qualitative analysis of parent-child interactions; math enthusiasm

Secondary Outcomes (explanation)

Experimental Design

The intent of the field experiments is to test several potential explanations for how digital apps and non-digital materials influence math learning. There are two main experiments. Experiment 1 asks whether digital apps are more efficient at conveying math skills and what role the parent plays in skill development. Experiment 2 asks how parents’ cognitive load affects children’s learning with digital and non-digital materials.

The experiments have the same general format in that parent-child pairs will come to the experimental site, consent to the experiment, be randomized to a treatment group or a control group, and participate in a 30–45-minute experiment. Children will be given a brief assessment of a specific math skill at the beginning and end of the experiment and the difference in score will be the main outcome for each experimental comparison. Each experiment will also assess children’s enthusiasm for learning, measured by whether when given the chance children choose to spend more time on another math activity. All experiments will be live coded by researchers and video recorded for auditing purposes.

We will survey parents at baseline and at the end of the intervention to collect data on family demographic characteristics, parents’ reports of the home learning environment, parents’ beliefs and expectations about their children’s math skill development, and parents’ expected return on their time and material investments in children’s math skill development. We will also ask questions that measure parents’ math anxiety and growth mindset and parents’ reports of the structural, behavioral, and cognitive barriers that stand in the way of their child’s math skill development. These will be used as covariates in the analysis of changes in assessment scores. We will manage the equal distribution of low vs. high income into those groups by stratifying on income within the randomization procedure.

Experimental Design Details

Randomization Method

Parent-child dyads recruited across 15 institutional sites were randomly assigned to a treatment condition immediately upon recruitment.

Randomization Unit

Our unit of randomization is parent-child dyads

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

We don't design clusters, so the planned number of clusters is 900 parent-child dyads.

Sample size: planned number of observations

900 parent-child dyads

Sample size (or number of clusters) by treatment arms

75 low/75 high-income parent-child dyads in each of the six treatment arms mentioned in the Intervention section

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Our minimum detectable effect size for the main Intention-to-Treat effect is designed to be 0.27 at the .05 level of statistical significance and with an 80 percent chance of deriving statistically significant impacts assuming a two-sided test of the null hypothesis.

Supporting Documents and Materials

IRB