NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
OpenTeQ - Opening the black box of Teacher Quality
Initial registration date
June 06, 2017
June 06, 2017 11:59 AM EDT
Milano Bicocca University
Other Primary Investigator(s)
Additional Trial Information
The quality of the teaching force is the key factor explaining the success of educational systems. We know that teachers differ widely regarding their impact on student achievement, but previous research failed in finding the sources of this variation. Educational economists consider teacher quality as a valuable and measurable feature, but they are not able to explain what generates it. Sociologists identify teacher quality as the consequence of individual and contextual characteristics. Pedagogies interpret it as a multidimensional concept and look at teaching practices and styles in daily classroom routines. This project wants to pool together these three perspectives. We argue that educational economists fail to explain the differences in teacher effectiveness because they look at teachers characteristics that are too far from the dynamic nature of teaching/learning process and because they do not consider the role played by contextual factors.
To prove the relevance of teacher relational skills in explaining the "mysterious" teacher quality, we are building up a collection of the best available tips to improve teacher relational skills, looking for them through international and national literature review and in depth interviews. More precisely, we producing a booklet facing six issues that teachers are called to face related to their relational skills. Each issue is treated not only in the booklet, but also with a very short video. We offered these tips to teachers in a sample of 100 lower secondary schools, randomized out of 198. Teacher used our booklet in the school year 2016/17 and, in within an action-research framework, we discuss the booklet and videos with them through registries on a platform.
We enrolled 198 lower secondary schools in 11 provinces (aggregated in 8 randomization blocks). An external researcher randomized school in the branches: two treatment groups and one control group.
The two treatment branches differ in the treatment delivery: in the first 50 schools the booklet was delivered individually only to Math and Italian teachers; in the last 50 schools, the booklet was delivered in a colletive meeting to all teachers in the school. The control group schools, randomly selected, will simply receive our intervention later, in spring 2018.
The RCT is designed in order to test the impact of our booklet on:
- teachers' well-being (in the short run, at the end of the first school year - CATI interviews)
- students' well-being (in the short run, at the end of the first school year - questionnaires only for 7th graders, due to budget constraints)
- students' achievement (in the short run, at the end of the first school year, and in the long run, at the end of the following school year - administrative data).
In addition, we want to test whether individual or collective delivery changes the intervention take up rate and its effectiveness.
The intervention is a professional development tool for teachers, enhaning their relational skills trough:
- a 30 pages booklet containing a collection of the best available tips to improve teacher relational skills, orgnanized in six issues that teachers are called to face daily (hoe to collaborate with colleagues; how to get a collaborative class climate; how to engage students; how to increase students' attention; how to handle troubling students; wow to establish positive relations with parents). The tips were collected both in scientific and grey literature and the booklet was validated through a Delphi process;
- access to a platform containing 1 introductory video and 6 contents video, treating each one a booklet issue;
- a registry were teachers interested in getting a certification were called to express their opinions about the tips and report their usage of the booklet contents.
Intervention Start Date
Intervention End Date
Primary Outcomes (end points)
In the short term (at the end of the first school year)
- teachers' well-being (measured through CATI interviews )
- students' well-being (measured through delf-administered questionnaires only for 7th graders, due to budget constraints)
- students' achievement (administrative data on students' marks).
In the long run (at the end of the following school years):
- students' achievement (administrative data on students' marks)
- students' performance in National standardized assessment (administrative data on students INVALSI test).
Primary Outcomes (explanation)
Teachers' and students' well-being measures are all indexes obtained from batteries of items, through the usual scaling procedures (exploratory and confirtmatory factor analyses). We are confident to get reliable measure, coinsidering that we mainly rely on previously validated constructs adapted to the Italian context and project aims.
Teachers' well-being measures:
- self efficacy
- relational satisfaction
- burnout - time management
- definition of features characterizing good performing teachers (three adjectives, coded after interview) Students' well-being measures:
- perceived classroom climate driven by teachers (teachers' support, task-oriented interaction, mutual respect, etc.)
- teachers' immediacy (verbal, not verbal, etc.)
- school engagement
- satisfaction about school life and improvement in school relations along the school year
- self efficacy and interest in Math and Italian - parental involvement
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
We enrolled 198 lower secondary schools in 11 provinces (aggregated in 8 randomization blocks). An external researcher randomized school in the branches whithin each block: two treatment groups (100 schools) and one control group (98 schools).
The two treatment branches differ in the treatment delivery: - in the first 50 schools the booklet was delivered individually only to Math and Italian teachers;
- in the last 50 schools, the booklet was delivered in a colletive meeting to all teachers in the school. The control group schools, randomly selected, will simply receive our intervention later, in spring 2018.
Experimental Design Details
Randomization done in office by a computer, by a researcher external to the project staff (Giovanni Abbiati - IRVAPP,
schools (whithin blocks)
Was the treatment clustered?
Sample size: planned number of clusters
198 schools whithin 8 blocks
Sample size: planned number of observations
around 2.200 teachers teaching Math or Italian to 7th graders
around 24.000 students for each grade (6th, 7th, 8th)
Sample size (or number of clusters) by treatment arms
50 schools individual treatment
50 schools collective treatment
98 schools control
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
* Please, note: conservative estimates
Teachers' outcomes - around 0,156 express in terms of effect size - both treatments considered together vs control
Teachers' outcomes - around 0,193 express in terms of effect size - each treatment considered separately vs control
Students' outcomes - around 0,084 express in terms of effect size - both treatments considered together vs control
Students' outcomes - around 0,104 express in terms of effect size - each treatment considered separately vs control
INSTITUTIONAL REVIEW BOARDS (IRBs)
Post Trial Information
Is the intervention completed?
Is data collection complete?