Do Time-Constraints Matter? How, Why, and for Whom?

Last registered on April 26, 2023

Pre-Trial

Trial Information

General Information

Title
Do Time-Constraints Matter? How, Why, and for Whom?
RCT ID
AEARCTR-0010719
Initial registration date
January 02, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 03, 2023, 5:32 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
April 26, 2023, 12:30 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Penn State University

Other Primary Investigator(s)

PI Affiliation
Penn State University
PI Affiliation
Bilkent University

Additional Trial Information

Status
On going
Start date
2022-10-12
End date
2024-02-28
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
We study test-taking behavior in time-constrained exams and describe time's role in shaping the distribution of students' performance outcomes. We do this by leveraging a policy change in Turkey which increased the college entrance exam time. Supporting administrative school-level data with the experimental data we collected, we show that having more test time is significantly associated with less wrong answers and less skipping patterns in a multiple-choice test where negative marking exists. Moreover, this association varies by gender and ability which we proxy for using background data. Female students are less likely to gain from an extra minute than male students and above median students are less likely to gain from an extra minute than below median students. Besides the performance changes, we also find that more test-time reduces the standard error and thus leads convergence of scores to each other. These data patterns motivates us to construct a model of test-taking behavior in time-constrained exam settings to firstly quantify the gains and losses from the policy change and secondly provide alternative exam policies with counterfactual exercises, interacting test-time with other test settings.
External Link(s)

Registration Citation

Citation
Akyol, Pelin , Kala Krishna and Esma Ozer. 2023. "Do Time-Constraints Matter? How, Why, and for Whom?." AEA RCT Registry. April 26. https://doi.org/10.1257/rct.10719-1.2
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2022-10-12
Intervention End Date
2023-02-28

Primary Outcomes

Primary Outcomes (end points)
We are interested in learning how time-constraints impact (i) probability of getting correct, (ii) probability of skipping in a multiple-choice exam set up.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The sample is composed of 12th grade students in a high school. There is one treatment-arm which is the (more) time limit in a multiple choice test.The sample is randomly grouped into two: time-relaxed and time-constrained.
In the first wave of experiment: students take a multiple choice (with 5 choices) test with four subjects (Turkish, Math, Science, Social Science) where negative marking applies. The booklets given to students are identical, i.e., the question difficulty/order doesn't differ across randomization units. Time-constrained group are given 90 minutes for 100 questions while time-relaxed group are given 170 minutes for 100 questions. The time-limit is designed such that the time-relaxed group is only ability-constrained. Proctoring is provided by randomly assigned school teachers in each of 5 classrooms. Students are incentivized with money per their ranking in their group. They can earn money between $0-30.
In the second wave of the experiment, all the settings stay same but the negative marking. We remove negative marking to see the changes in our primary outcomes with respect to time-constraints.
Experimental Design Details
Randomization Method
Randomization is done using computer with random number generator.
Randomization Unit
Randomization unit is individiual in a given school.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
One school with planned 80-100 students.
Sample size: planned number of observations
80-100 students.
Sample size (or number of clusters) by treatment arms
40-50 students control, 40-50 students treatment.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Pennsylvania State University
IRB Approval Date
2022-09-14
IRB Approval Number
STUDY00020658

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials