Improving Teachers’ Equitable Mathematics Instruction Through Integrating Automated Feedback and Coaching: A Randomized Controlled Trial

Last registered on September 24, 2024

Pre-Trial

Trial Information

General Information

Title
Improving Teachers’ Equitable Mathematics Instruction Through Integrating Automated Feedback and Coaching: A Randomized Controlled Trial
RCT ID
AEARCTR-0014383
Initial registration date
September 17, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 24, 2024, 2:45 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
University of Maryland

Other Primary Investigator(s)

PI Affiliation
Harvard University
PI Affiliation
Stanford University

Additional Trial Information

Status
On going
Start date
2024-07-01
End date
2026-07-01
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Embedding artificial intelligence-based automated feedback in teacher coaching can enhance teacher effectiveness with mathematics curriculum materials. This project will provide all teachers in this study with automated feedback and coaching on curriculum-aligned, equity-focused instructional moves. While doing so, we will vary the approach taken by this coaching to evaluate whether reflective or directive coaching is more effective in changing instructional quality and, down the line, student outcomes. This project seeks to evaluate these two models of instructional coaching by integrating state-of-the-art automated feedback on mathematics lessons with coaching routines that support teacher change. The M-Powering Teachers automated feedback tool provides teachers with feedback related to instructional moves such as focusing questions, uptake of student ideas, student idea attribution, and more. Our central goal is to design feedback and coaching experiences that maximize impact on teaching and learning, especially for historically marginalized student populations.
External Link(s)

Registration Citation

Citation
Demszky, Dora, Heather Hill and Jing Liu. 2024. "Improving Teachers’ Equitable Mathematics Instruction Through Integrating Automated Feedback and Coaching: A Randomized Controlled Trial." AEA RCT Registry. September 24. https://doi.org/10.1257/rct.14383-1.0
Experimental Details

Interventions

Intervention(s)
In this study, we will explore the efficacy of two coaching approaches, reflective and directive, through a randomized controlled trial. In the reflective coaching model, teacher self-reflection guides the coaching conversation. Goker (2021) defines reflection as “a reaction to past experience and includes deliberate recall and analysis of that experience and decision-making and as a reference for further plans and actions” (p. 425). By giving teachers the space to think critically about their practice and to take the lead in analyzing their own instruction, teachers feel empowered to make changes to their practice. Literature on adult learning theory shows that self-direction is crucial for adults to change their behavior (Knowles et al., 2011; Merriam, 2001). Soisangwarn and Wongwanich (2014) find that teacher self-reflection through peer coaching improves teachers’ motivation to improve their teaching skills in Thailand. Additionally, a study of pre-service teachers in Turkey shows that reflective coaching improves teachers’ instructional skills in lesson planning and implementation (Goker, 2021).

Because teachers’ self-reflections are sometimes inaccurate and biased (e.g., teachers might think they are using an instructional talk move more than they are in reality), some recommend a more directive approach to coaching (Hammond & Moore, 2018). In the directive coaching model, the instructional coach leads the conversation with the teacher by providing more explicit guidance and specific advice. Ippolito (2010) defines directive coaching as “coaching for the implementation of particular practices” (p. 164). In the directive approach, the instructional coach takes the role of the expert to guide the teacher toward executing certain instructional moves with less teacher input than the reflective approach. Directive coaching may be more beneficial for novice teachers, while responsive coaching may foster more trusting relationships between the teacher and coach (Ippolito, 2010). Both coaching models utilize structured routines to encourage PL, but the approaches vary in the level of coach direction and teacher self-reflection. In practice, many coaching conversations incorporate both approaches, and many coaches switch between the two models depending on the teacher and the context of the PL (Ippolito, 2010). While both approaches seem valuable, determining which coaching model is more successful in improving effective and equitable mathematics instruction will allow practitioners to maximize PL impacts in the future.
Intervention Start Date
2024-09-01
Intervention End Date
2026-06-30

Primary Outcomes

Primary Outcomes (end points)
Teacher practices including teacher uptake of student ideas, focusing questions, student reasoning; student responses to surveys about engagement and belonging; teacher responses to surveys
Primary Outcomes (explanation)
Teacher practices are all created by applying natural language processing to transcripts of classroom recordings

Secondary Outcomes

Secondary Outcomes (end points)
Student academic achievement
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Half of teachers will be assigned to reflective coaching, and the other half will be assigned to directive coaching. Randomization will be done within coaches.
Experimental Design Details
Not available
Randomization Method
Stata (stratarand)
Randomization Unit
Coach
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
We will have 15-25 coaches. We will complete this study in different waves, so some coaches will participate in multiple waves.
Sample size: planned number of observations
180 teachers, assuming 25 students per teacher (4500 students)
Sample size (or number of clusters) by treatment arms
90 teachers in the reflective condition and 90 teachers in the directive condition
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
University of Maryland, College Park, Institutional Review Board
IRB Approval Date
2024-06-07
IRB Approval Number
2194458