Using Adaptive Learning to Remedy Educational Disadvantage

Last registered on September 12, 2024
View Trial History

Pre-Trial

Trial Information

General Information

Title
Using Adaptive Learning to Remedy Educational Disadvantage
RCT ID
AEARCTR-0014180
Initial registration date
August 28, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 12, 2024, 4:32 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
United Kingdom of Great Britain and Northern Ireland
Region
South

Primary Investigator

Name
Thomas Gall
Affiliation
University of Southampton
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Michael Vlassopoulos
PI Affiliation
University of Southampton
Contact Investigator
PI Name
Chris Downey
PI Affiliation
University of Southampton
Contact Investigator
PI Name
Benjamin Davies
PI Affiliation
Univeristy of Southampton
Contact Investigator

Additional Trial Information

Status
In development
Start date
2023-09-01
End date
2028-09-30
Keywords
Education
Additional Keywords
Higher Education; Adaptive Learning
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
In the UK, access to many university courses, especially those associated to high incomes, depends on the A levels taken. These in turn reflect a choice made at school age, depending on one’s environment at school and the quality of advice available. A planned change in economics admissions criteria at a Russell Group university will extend access to applicants without mathematics A levels, broadening the set of potential students in terms of secondary education choices and possibly socio-economic backgrounds. We examine potential changes in the demographics of economics students, their academic and labour market outcomes, and measures of social mobility.
External Link(s)

Registration Citation

Citation
Davies, Benjamin et al. 2024. "Using Adaptive Learning to Remedy Educational Disadvantage." AEA RCT Registry. September 12. https://doi.org/10.1257/rct.14180-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
We aim to study effects of broadening university admission criteria for a quantitative degree programme, supported by adapting teaching and learning methods to introduce adaptive learning in quantitative modules.
Intervention Start Date
2025-01-27
Intervention End Date
2028-09-30

Primary Outcomes

Primary Outcomes (end points)
Academic outcomes: module and course marks
Labour Market Outcomes: employment status and wage six months after graduation
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Demographics of incoming students: Gender, ethnicity, geographic origin, school marks (A levels), SES such as POLAR4, parental education
Indicators of student well-being
Secondary Outcomes (explanation)
Indicators of student well-being are elicited through focus group interviews and include measures of agency, anxiety, workload, sense of achievement and similar.

Experimental Design

Experimental Design
Stage 1: Diagnostics. Evaluating the predictive power of different diagnostic tests for incoming students for later academic performance, using module marks and diagnostic tests in later semesters and years as outcomes. Baseline cohort in academic year 2023/2024.

Stage 2: Adaptive Learning. Adaptive learning methods will be implemented in Statistics for Economics in S2 2024/2025, in Mathematics for Economics in S1 2025/2026 for the foloowing cohort.

Stage 3: Expanding Access. From 2026/2027 the university plans to expand access by changing admission criteria to no longer require A level Mathematics, for some courses, allowing for within and across cohort comparisons and causal inference, given a host of demographic control variables.
Experimental Design Details
Not available
Randomization Method
Staggered introduction of interventions:
- 2023/2024: No intervention
- 2024/2025: Semester 2 teaching intervention
- 2025/2026: Semester 1 and Semester 2 teaching interventions
- 2026/2027: Semester 1 and Semester 2 teaching interventions and admissions intervention
- 2027/2028: Semester 1 and Semester 2 teaching interventions and admissions intervention
Randomization Unit
Cohort of students on particular programmes
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
5 cohorts of students
Sample size: planned number of observations
Total sample size will be about 1,800 students
Sample size (or number of clusters) by treatment arms
For Semester 2 teaching intervention: two cohorts control, four cohorts intervention (including within subject variation)
For Semester 1 teaching intervention: two cohorts control, three cohorts intervention
For Admissions intervention: three cohorts control, two cohorts intervention
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
For the adaptive learning treatments the primary outcome is the maths mark (mean 59.4 std dev 14.6), with alpha=0.05 beta=0.2 minimal detectable effect size is 3 or 5%. For the admission treatment a primary outcome is POLAR4 (mean 4.12 std dev 1.15) with alpha=0.05 beta=0.2 minimal detectable effect size is 0.24, or 6%.
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Faculty Ethics Committee, Faculty of Social Sciences, University of Southampton
IRB Approval Date
2024-08-21
IRB Approval Number
83113.A1
Analysis Plan