About TIME (Technology in Math Education): RCT

Last registered on January 30, 2023

View Trial History

Pre-Trial

Trial Information

General Information

Title

About TIME (Technology in Math Education): RCT

RCT ID

AEARCTR-0009358

Initial registration date

May 05, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

May 05, 2022, 12:57 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated

January 30, 2023, 12:23 PM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country

United States of America

Region

Chicago

Primary Investigator

Name

Haoxuan Liu

Affiliation

University of Chicago

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Ariel Kalil

PI Affiliation

University of Chicago

Contact Investigator

PI Name

Susan Mayer

PI Affiliation

University of Chicago

Contact Investigator

Additional Trial Information

Status

In development

Start date

2022-12-01

End date

2023-05-15

Keywords

Behavior, Education, Other

Additional Keywords

Parent engagement, family engagement, poverty and inequality, math, early childhood, preschool

JEL code(s)

Secondary IDs

Prior work

This trial does not extend or rely on any prior RCTs.

Abstract

About TIME (Technology in Math Education) explores how technology can best be used to promote the acquisition of math skills among young children and reduce the gap in skill development between high- and low-income children. We hypothesize that digital apps may improve young children’s math skills over the long run compared to similar non-digital learning materials and the business-as-usual scenario both because children are likely to spend more time using digital apps, and digital apps may be more efficient at conveying math skills. About TIME will proceed in two parts. The first is a series of field experiments to help us understand how technology works to promote learning. The second is a large-scale randomized controlled trial intended to measure the increase in math skills among preschool-age children who are provided with a tablet containing high-quality math apps and a group of children provided with high-quality non-digital materials of the sort recommended by experts in math learning and intended to promote the same skills as the digital apps (corresponding to this registration). We will have a control group without intervention. The results of the About TIME Project will: 1. compare the increase in math skills for children across three groups, 2. estimate the extent to which parent characteristics and barriers to parent engagement influence the effect of both digital and non-digital learning tools in building child math skill, 3. estimate how parent involvement moderates the effect of digital apps and non-digital materials on children’s math skills, 4. estimate how digital apps and non-digital materials affect children’s and parents’ enthusiasm for math, 5. describe which apps are used the most, when they are used, and which features of apps are most often used by children.

External Link(s)

Registration Citation

Citation

Kalil, Ariel, Haoxuan Liu and Susan Mayer. 2023. "About TIME (Technology in Math Education): RCT." AEA RCT Registry. January 30. https://doi.org/10.1257/rct.9358-1.3

Sponsors & Partners

Experimental Details

Interventions

Intervention(s)

Intervention (Hidden)

Intervention Start Date

2022-12-01

Intervention End Date

2023-05-15

Primary Outcomes

Primary Outcomes (end points)

Overall effectiveness (a combination of time on task and efficiency of time on task) of digital and non-digital math learning materials in conveying numeracy skills relative to the control for preschool-aged children analyzed by parent socioeconomic status

Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)

Changes in parent attitudes and beliefs that are known to predict children's math skills, math enthusiasm, reported usage of non-digital materials, recorded usage of digital materials, and reported engagement in other math talk/learning activities

Secondary Outcomes (explanation)

Experimental Design

Once recruited, families will provide consent to participate in the RCT and then be randomly assigned to one of two treatment groups or to a control group. Parents in one treatment group will receive a tablet loaded with high-quality math apps. Parents in the other treatment group will receive high-quality non-digital materials that are designed to convey the same skills as the apps. The control group is a “business as usual” group that will receive no treatment other than nominal materials unrelated to early math learning such as coloring books. We will survey parents at baseline and at the end of the intervention to collect data on family demographic characteristics, parents’ reports of the home learning environment, parents’ beliefs and expectations about their children’s math skill development, and parents’ expected return on their time and material investments in children’s math skill development. We will also ask questions that measure parents’ math anxiety and growth mindset and parents’ reports of the structural, behavioral, and cognitive barriers that stand in the way of their child’s math skill development. These will be used as covariates in the analysis of changes in assessment scores. We will manage the equal distribution of low vs. high income into those groups by stratifying on income within the randomization procedure.

Experimental Design Details

Randomization Method

We conduct the simple randomization procedure after all participants are recruited and prior to the start of the intervention using a statistical software program.

Randomization Unit

Our unit of randomization is parent-child dyads.

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

We don't design clusters, so the planned number of clusters is 700 parent-child dyads.

Sample size: planned number of observations

700 parent-child dyads

Sample size (or number of clusters) by treatment arms

Around 117 low/117 high-income parent-child dyads will be assigned into one of two treatment groups or to a control group.

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Our minimum detectable effect size for the main Intention-to-Treat effect is designed to be 0.2 at the .05 level of statistical significance and with an 80 percent chance of deriving statistically significant impacts assuming a two-sided test of the null hypothesis.

Supporting Documents and Materials

IRB