x

NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
Can Teacher Peer Feedback Improve Test Scores? Evidence from an RCT in 182 Schools
Last registered on November 24, 2016

Pre-Trial

Trial Information
General Information
Title
Can Teacher Peer Feedback Improve Test Scores? Evidence from an RCT in 182 Schools
RCT ID
AEARCTR-0001779
Initial registration date
November 24, 2016
Last updated
November 24, 2016 9:16 AM EST
Location(s)
Primary Investigator
Affiliation
Other Primary Investigator(s)
PI Affiliation
Additional Trial Information
Status
On going
Start date
2013-05-01
End date
2017-05-01
Secondary IDs
Abstract
It is widely accepted that teachers are the most important factor in school effectiveness. However, there is little robust quantitative evidence of the effectiveness of programmes that improve teacher quality. Of reliable research carried out on teacher development, knowledge based training has typically been found to be ineffective, while programmes that involve teacher observations that are embedded into the school alongside effective feedback have been found to have effects (Taylor and Taylor, 2013).
Lesson Study is a programme incorporating teacher observations and feedback. It is a professional development programme with a long history of use in Japan and is being increasingly used in the UK. Teachers work in small groups to plan lessons that address a shared teaching and learning goal. They then observe each other’s lessons, focusing on pupil learning rather than the teacher. They then discuss the lesson, refine the lesson plans and repeat the process. In the planning, two 'case pupils' are selected that are typical of a group of pupils in the class, so that the impact of the lessons on pupils with identified barriers to learning can be monitored. After the lesson has taken place, it is discussed, analysed and evaluated by the group of teachers, leading to revisions being made in content, structure and delivery. This process includes feedback from the case pupils in the form of a teacher/pupil interview. The knowledge generated by the whole process is then shared more widely with other colleagues.
An evaluation of the UK’s National Strategies’ Leading Teachers Programme, which involved Lesson Study, showed that those schools using this approach (among others) out-performed a comparison group in both English and Mathematics. Lesson Study also shares many of the key characteristics of effective CPD that were identified in a systematic review produced by the Evidence for Policy and Practice Information and Co-ordinating Centre. Research investigating the National Strategies’ Leading Teachers Programme (Hadfield, Jopling and Emira, 2011), of which Lesson Study was a key element, revealed positive impacts on pupil outcomes. However, this study did not take account of potential differences in the treatment and control groups used in the study, nor was it able to follow up the long-term impacts of the programme.
The trial of Lesson Study incorporated Talk for Learning, an approach which aims to improve the quality of classroom talk in order to increase pupils' engagement, learning and attainment. Participating schools were asked to deliver Lesson Study within literacy and maths lessons using Talk for Maths and Talk for Literacy with teachers trained in specific approaches for both. The trial was therefore a test of the effectiveness of this method as well as of the Lesson Study approach.
External Link(s)
Registration Citation
Citation
Murphy, Richard and Felix Weinhardt. 2016. "Can Teacher Peer Feedback Improve Test Scores? Evidence from an RCT in 182 Schools." AEA RCT Registry. November 24. https://doi.org/10.1257/rct.1779-1.0.
Former Citation
Murphy, Richard, Felix Weinhardt and Felix Weinhardt. 2016. "Can Teacher Peer Feedback Improve Test Scores? Evidence from an RCT in 182 Schools." AEA RCT Registry. November 24. http://www.socialscienceregistry.org/trials/1779/history/11998.
Experimental Details
Interventions
Intervention(s)
It is widely accepted that teachers are the most important factor in school effectiveness. However, there is little robust quantitative evidence of the effectiveness of programmes that improve teacher quality. Of reliable research carried out on teacher development, knowledge based training has typically been found to be ineffective, while programmes that involve teacher observations that are embedded into the school alongside effective feedback have been found to have effects (Taylor and Taylor, 2013).
Lesson Study is a programme incorporating teacher observations and feedback. It is a professional development programme with a long history of use in Japan and is being increasingly used in the UK. Teachers work in small groups to plan lessons that address a shared teaching and learning goal. They then observe each other’s lessons, focusing on pupil learning rather than the teacher. They then discuss the lesson, refine the lesson plans and repeat the process. In the planning, two 'case pupils' are selected that are typical of a group of pupils in the class, so that the impact of the lessons on pupils with identified barriers to learning can be monitored. After the lesson has taken place, it is discussed, analysed and evaluated by the group of teachers, leading to revisions being made in content, structure and delivery. This process includes feedback from the case pupils in the form of a teacher/pupil interview. The knowledge generated by the whole process is then shared more widely with other colleagues.
An evaluation of the UK’s National Strategies’ Leading Teachers Programme, which involved Lesson Study, showed that those schools using this approach (among others) out-performed a comparison group in both English and Mathematics. Lesson Study also shares many of the key characteristics of effective CPD that were identified in a systematic review produced by the Evidence for Policy and Practice Information and Co-ordinating Centre. Research investigating the National Strategies’ Leading Teachers Programme (Hadfield, Jopling and Emira, 2011), of which Lesson Study was a key element, revealed positive impacts on pupil outcomes. However, this study did not take account of potential differences in the treatment and control groups used in the study, nor was it able to follow up the long-term impacts of the programme.
The trial of Lesson Study incorporated Talk for Learning, an approach which aims to improve the quality of classroom talk in order to increase pupils' engagement, learning and attainment. Participating schools were asked to deliver Lesson Study within literacy and maths lessons using Talk for Maths and Talk for Literacy with teachers trained in specific approaches for both. The trial was therefore a test of the effectiveness of this method as well as of the Lesson Study approach.
Intervention Start Date
2013-09-02
Intervention End Date
2015-07-20
Primary Outcomes
Primary Outcomes (end points)
Student performance (for details see uploaded SAP document)
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
In order to have sufficient schools to meet the power calculations the trial was designed to maximise sample size by keeping costs to a minimum. This was achieved by relying on Key Stage 2 (KS2) test score data from the treatment and control schools. Once a school was assigned as a control school they received a letter stating that as a result of the randomisation process they had not been selected, and we had no further contact with them for the duration of the experiment. We are still able to collect outcome data on the students from the control schools by using the national pupil database.
Schools that were selected into the treatment group were asked to select three teachers to be part of the Lesson Study trial. These teachers were all to come from years 4 and 5. Where that was not possible, because there was only one class per cohort for example, schools were free to involve teachers from Years 3 or 6. This means that is possible for students to be taught by Lesson Study teachers from Year 3 through to Year 6.
Table 1 below shows the targeted cohorts in academic Years 4 and 5 during the calendar school years 2013-14 and 2014-15. The outcome is measured at the end of Year 6. C1 and C2 sat their respective KS2 exams in the summer of 2015-16 and 2015-16 respectfully. C1 will only have one year of treatment, C2 will likely have two years of treatment, but both will have a year without treatment before their KS2 examinations. However, for reasons explained above it will also be the case that some students will be treated in schools during Year 6. These students could have taken their KS2 examinations in 2013-14, 2014-15, or 2015-16. Moreover there are likely to be students taught by Lesson Study Teachers, for whom we would not have observed their KS2 outcomes by the summer of 2015-16.

Table 1
2013-14 2014-15 2015-16 2016-17 2017-18
Year 2 C4 (KS1 exam)
Year 3 C3 C4
Year 4 C2 C3 C4 .. ..
Year 5 C1 C2 C3 C4 ..
Year 6 C0 C1 C2 C3 C4 (KS2 exam)

Therefore we will be collecting outcome measures for three cohorts of students (C0, C1 and C2) in both the treatment and control schools. We will also collect these students KS1 test scores from the NPD to use as baseline measures of achievement. Cohorts C1, C2 and C3 all sat their KS1 exams before the intervention took place, so these test outcomes can serve as baseline even in the presence of across-year spillovers of the treatment within schools.
Experimental Design Details
Randomization Method
Randomization done in office by a computer setting different seeds
Randomization Unit
Schools
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
160 schools
Sample size: planned number of observations
160 schools times number of students: exact number unknown since the trial targets teachers and we only get access to baseline as well as outcome data through administrative national exams. There is no baseline or outcomes data collection done by ourselves. Instead, this information is gathered through ex-post linkeage with administrative test score records.
Sample size (or number of clusters) by treatment arms
This resulted in a total of 89 schools assigned to treatment, 92 to control and one was left unassigned.
Details see uploaded consort diagram in analysis plan.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The study only detects true effect sizes of 0.1 s.d. in the outcome with a probability of about 80 percent. Details in uploaded SAP
Supporting Documents and Materials

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
LSE Research Governance Board
IRB Approval Date
2016-05-25
IRB Approval Number
273-472
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports, Papers & Other Materials
Relevant Paper(s)
REPORTS & OTHER MATERIALS