Impact and Mechanisms: Why Consulting Matters in Human Capital Intensive Organizations. Evidence from a Field Experiment on Teacher Coaching.

Last registered on September 13, 2017

Pre-Trial

Trial Information

General Information

Title
Impact and Mechanisms: Why Consulting Matters in Human Capital Intensive Organizations. Evidence from a Field Experiment on Teacher Coaching.
RCT ID
AEARCTR-0002419
Initial registration date
September 13, 2017

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 13, 2017, 3:13 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Nantes Université

Other Primary Investigator(s)

PI Affiliation
Aarhus University & Trygfonden's Centre for Child Research
PI Affiliation
Aarhus University & Trygfonden's Centre for Child Research

Additional Trial Information

Status
On going
Start date
2017-08-09
End date
2019-12-31
Secondary IDs
Abstract
Recent studies have demonstrated substantial effects of consultancy on learning and productivity (Allen et al., 2011; Bloom et al., 2013; Hanna et al., 2014). However, the channels through which the effect of consultancy materializes and the relative strength of the different channels remain to be clarified. In order to study this question, we implement a randomized controlled trial investigating the impact of consultancy in the form of a one-year coaching program aiming to improve teacher skills through peer coaching. It will allow us to measure the relative strength of various channels through which the effects of consulting can materialize.
External Link(s)

Registration Citation

Citation
Andersen, Simon Calmar, Bastien Michel and Helena Skyt Nielsen. 2017. "Impact and Mechanisms: Why Consulting Matters in Human Capital Intensive Organizations. Evidence from a Field Experiment on Teacher Coaching.." AEA RCT Registry. September 13. https://doi.org/10.1257/rct.2419-1.0
Former Citation
Andersen, Simon Calmar, Bastien Michel and Helena Skyt Nielsen. 2017. "Impact and Mechanisms: Why Consulting Matters in Human Capital Intensive Organizations. Evidence from a Field Experiment on Teacher Coaching.." AEA RCT Registry. September 13. https://www.socialscienceregistry.org/trials/2419/history/21418
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
As part of this study, we aim to measure the impact of a teacher-coaching intervention. We will do so in a context where all teachers (whether or not they receive the coaching intervention) will benefit from a professional development course designed to improve their teaching skills. Half of those teachers will be selected to benefit from a coaching intervention as well.

3.1 Professional development course
We will implement an intervention where all participating teachers and coaches attend five one-day course modules on the following education production factors: i) Physical surroundings, ii) Structuring schoolwork, iii) Self-regulation iv) The teacher's focus, and v) The teacher's communication. As part of these course modules, teachers will learn to use both technological input factors (e.g. timer, checklist, five-point scale, etc.) and behavioral input factors (e.g. support in transition phases, use calm voice, mentalizing, etc.). Furthermore, during each module, teachers will be asked to reflect on their own students and to set an agenda for themselves detailing how they intend to change their teaching practices so as to take into account what they will have learnt during the course modules. In addition, each course module will start out with rehearsal and reflection on what they have learnt in order to consolidate knowledge.

The course modules will be held on the following dates:
- Course module 1: August 9th, 2017
- Course module 2: September 14th, 2017
- Course module 3: October 31st, 2017
- Course module 4: November 28th, 2017
- Course module 5: February 7th, 2017

3.2 Treatment: Teacher-coaching program
Teachers selected to receive the teacher-coaching program will be paired with a co-teacher, their "coach", with whom they will spend 2.5 hours per week during an entire school year. Co-teachers will help teachers get closer to the production-possibility frontier through "peer coaching", as part of which the teacher receives feedback on how to improve teaching practice. These sessions will be dedicated to the implementation of what will have been covered during the professional development courses.
Intervention (Hidden)
Intervention Start Date
2017-08-09
Intervention End Date
2018-06-30

Primary Outcomes

Primary Outcomes (end points)
In order to answer our research questions, several rounds of data collection will be carried out (for each batch) at different points in time:
1/ At the beginning of the school year, during the first 20 minutes of the first course module (baseline data collection, August 2017 - before the results of the randomization are shared with the schools)
2/ During the first 20 minutes of each remaining course module (follow-up surveys)
3/ After the first coaching session carried out after each course module (monitoring data collection)
4/ At the end of the school year, upon completion of the course modules (endline data collection, June 2018)

The data will be gathered through different data sources:
- Administrative registers: student, parent and teacher background, tracking of students and teachers, output measures (e.g. national test scores and grades for the teacher's past, current, and future students)
- Observation data: classroom instrument developed from the course material and/or CLASS-UE measurement
- Teachers and students surveys: they will be collected online using the SurveyXact platform

Monitoring data
We will collect information depicting how the intervention is carried out. In particular, we will collect information on the identity of the teachers attending the professional development course and on those actually receiving the coaching intervention.
Furthermore, descriptive information will also be collected so as to understand how the coaching intervention is implemented. Teachers will be asked to answer (together with their coach) a short questionnaire after the first coaching session carried out after each course module. This questionnaire will collect information on their experience with the co-teaching intervention.

Outcome variables
Our primary outcome is teachers' teaching skills and, more specifically, whether or not they use the elements taught during the professional development course in class. This will be assessed by surveyors who will attend teachers' class and will rate the extent to which they use each of the techniques taught during the course. We will supplement these class observations by a measure of teaching skills as reported by the students using a validated index of teaching quality (TRIPOD).

Our secondary outcome is students' achievements and, in particular, their reading skills (as measured by national standardized reading tests) and socio-emotional skills (as measured by national standardized wellbeing surveys).
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
As there are strong reasons to expect significant selection with respect to the type of teachers who enroll in a coaching program, we will measure its impact using a randomized controlled trial design.

The coaching intervention will be randomized at the teacher-level. For implementation-related reasons, it was decided that the draw would be stratified by school. As a consequence, we will randomly select in each school half of the language arts teachers to benefit from the coaching intervention. In schools where an uneven number of teachers will be enrolled in the experiment, the number of teachers who should benefit from the coaching intervention was decided in advance for logistical reasons. We will adjust for the ensuing variation in teachers' probability to receive the coaching intervention across schools in our estimation strategy.

When schools will have appointed more than one coach, the teacher-coach pairs will be randomly generated.
Experimental Design Details
Randomization Method
Randomization done in office by a computer.
Randomization Unit
The coaching intervention will be randomized at the language arts teacher-level. Again, for implementation-related reasons, it was decided that the draw would be stratified by school.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
17 schools
Sample size: planned number of observations
68 teachers 1490 students
Sample size (or number of clusters) by treatment arms
In each school, 50% of the teachers enrolled in the experiment will benefit from the intervention.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Our power analysis suggests that with a power of 80% we would be able to detect a minimum effect size of 0.50 to 0.60 standard deviations in teachers' behavior and a minimum effect size of 0.23 to 0.30 standard deviations in students' test scores.
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials