Back to History Current Version

Across-test human performance in mathematics

Last registered on October 17, 2023
View Trial History

Pre-Trial

Trial Information

General Information

Title
Across-test human performance in mathematics
RCT ID
AEARCTR-0012253
Initial registration date
October 06, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 17, 2023, 10:55 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
United States of America
Region

Primary Investigator

Name
Raphael Raux
Affiliation
Harvard University
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Bnaya Dreyfuss
PI Affiliation
Harvard University
Contact Investigator

Additional Trial Information

Status
In development
Start date
2023-10-06
End date
2024-09-30
Keywords
Behavior, Lab
Additional Keywords
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
We study how human subjects perform on a series of math questions sampled from different standardized tests of various difficulty. The study will take place online and recruit subjects from the Prolific platform.
External Link(s)

Registration Citation

Citation
Dreyfuss, Bnaya and Raphael Raux. 2023. "Across-test human performance in mathematics." AEA RCT Registry. October 17. https://doi.org/10.1257/rct.12253-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
We administer a math test online, where subjects attempt to solve 30 randomly-sample questions from 3 standardized tests of various difficulty levels (4th grade, 8th grade, and High school). These tests belong to the TIMSS family of tests, which are administered every four years in several countries.
Intervention (Hidden)
Intervention Start Date
2023-10-06
Intervention End Date
2024-09-30

Primary Outcomes

Primary Outcomes (end points)
Subject performance in the test.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Subjects see a total of 30 questions, split into 3 blocks of 10. Questions are randomly sampled from the total pool of questions, and block order is randomized. For this stage there are no "treatments".

We expect subjects' performance to be decreasing on average across test levels. At the extremes of the ability distribution, a flatter slope can be expected given that performance (defined as success rate across questions seen) is bounded between 0 and 1.
We also expect that within-test performance will decrease with question difficulty, proxied by the % of students who answered the question correctly.
Experimental Design Details
Randomization Method
Randomization done through qualtrics survey flow (for blocks) and block randomization (for questions within a block).
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
Between 200 and 350 subjects.
Sample size: planned number of observations
Between 200 and 350 subjects.
Sample size (or number of clusters) by treatment arms
Between 200 and 350 subjects.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Committed on the Use of Human Subjects - Harvard
IRB Approval Date
2023-06-02
IRB Approval Number
IRB23-0588
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials