OpenTeQ - Opening the black box of Teacher Quality

Last registered on June 06, 2017

View Trial History

Pre-Trial

Trial Information

General Information

Title

OpenTeQ - Opening the black box of Teacher Quality

RCT ID

AEARCTR-0001638

Initial registration date

June 06, 2017

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

June 06, 2017, 11:59 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country

Italy

Region

Country

Italy

Region

Primary Investigator

Name

Gianluca Argentin

Affiliation

Milano Bicocca University

Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status

On going

Start date

2016-03-01

End date

2018-09-22

Keywords

Education

Additional Keywords

teacher effectiveness, soft skills, relational skills

JEL code(s)

Secondary IDs

Abstract

The quality of the teaching force is the key factor explaining the success of educational systems. We know that teachers differ widely regarding their impact on student achievement, but previous research failed in finding the sources of this variation. Educational economists consider teacher quality as a valuable and measurable feature, but they are not able to explain what generates it. Sociologists identify teacher quality as the consequence of individual and contextual characteristics. Pedagogies interpret it as a multidimensional concept and look at teaching practices and styles in daily classroom routines. This project wants to pool together these three perspectives. We argue that educational economists fail to explain the differences in teacher effectiveness because they look at teachers characteristics that are too far from the dynamic nature of teaching/learning process and because they do not consider the role played by contextual factors.

To prove the relevance of teacher relational skills in explaining the "mysterious" teacher quality, we are building up a collection of the best available tips to improve teacher relational skills, looking for them through international and national literature review and in depth interviews. More precisely, we producing a booklet facing six issues that teachers are called to face related to their relational skills. Each issue is treated not only in the booklet, but also with a very short video. We offered these tips to teachers in a sample of 100 lower secondary schools, randomized out of 198. Teacher used our booklet in the school year 2016/17 and, in within an action-research framework, we discuss the booklet and videos with them through registries on a platform.

We enrolled 198 lower secondary schools in 11 provinces (aggregated in 8 randomization blocks). An external researcher randomized school in the branches: two treatment groups and one control group.
The two treatment branches differ in the treatment delivery: in the first 50 schools the booklet was delivered individually only to Math and Italian teachers; in the last 50 schools, the booklet was delivered in a colletive meeting to all teachers in the school. The control group schools, randomly selected, will simply receive our intervention later, in spring 2018.

The RCT is designed in order to test the impact of our booklet on:
- teachers' well-being (in the short run, at the end of the first school year - CATI interviews)
- students' well-being (in the short run, at the end of the first school year - questionnaires only for 7th graders, due to budget constraints)
- students' achievement (in the short run, at the end of the first school year, and in the long run, at the end of the following school year - administrative data).
In addition, we want to test whether individual or collective delivery changes the intervention take up rate and its effectiveness.

External Link(s)

intervention platform

project description

Registration Citation

Citation

Argentin, Gianluca. 2017. "OpenTeQ - Opening the black box of Teacher Quality." AEA RCT Registry. June 06. https://doi.org/10.1257/rct.1638-1.0

Former Citation

Argentin, Gianluca. 2017. "OpenTeQ - Opening the black box of Teacher Quality." AEA RCT Registry. June 06. https://www.socialscienceregistry.org/trials/1638/history/18306

Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Experimental Details

Interventions

Intervention(s)

The intervention is a professional development tool for teachers, enhaning their relational skills trough:
- a 30 pages booklet containing a collection of the best available tips to improve teacher relational skills, orgnanized in six issues that teachers are called to face daily (hoe to collaborate with colleagues; how to get a collaborative class climate; how to engage students; how to increase students' attention; how to handle troubling students; wow to establish positive relations with parents). The tips were collected both in scientific and grey literature and the booklet was validated through a Delphi process;
- access to a platform containing 1 introductory video and 6 contents video, treating each one a booklet issue;
- a registry were teachers interested in getting a certification were called to express their opinions about the tips and report their usage of the booklet contents.

Intervention Start Date

2016-10-20

Intervention End Date

2017-07-15

Primary Outcomes

Primary Outcomes (end points)

In the short term (at the end of the first school year)
- teachers' well-being (measured through CATI interviews )
- students' well-being (measured through delf-administered questionnaires only for 7th graders, due to budget constraints)
- students' achievement (administrative data on students' marks).

In the long run (at the end of the following school years):
- students' achievement (administrative data on students' marks)
- students' performance in National standardized assessment (administrative data on students INVALSI test).

Primary Outcomes (explanation)

Teachers' and students' well-being measures are all indexes obtained from batteries of items, through the usual scaling procedures (exploratory and confirtmatory factor analyses). We are confident to get reliable measure, coinsidering that we mainly rely on previously validated constructs adapted to the Italian context and project aims.

Teachers' well-being measures:
- self efficacy
- relational satisfaction
- burnout
- time management
- definition of features characterizing good performing teachers (three adjectives, coded after interview)

Students' well-being measures:
- perceived classroom climate driven by teachers (teachers' support, task-oriented interaction, mutual respect, etc.)
- teachers' immediacy (verbal, not verbal, etc.)
- school engagement
- satisfaction about school life and improvement in school relations along the school year
- self efficacy and interest in Math and Italian
- parental involvement

Secondary Outcomes

Secondary Outcomes (end points)

Secondary Outcomes (explanation)

Experimental Design

We enrolled 198 lower secondary schools in 11 provinces (aggregated in 8 randomization blocks).
An external researcher randomized school in the branches whithin each block: two treatment groups (100 schools) and one control group (98 schools).
The two treatment branches differ in the treatment delivery:
- in the first 50 schools the booklet was delivered individually only to Math and Italian teachers;
- in the last 50 schools, the booklet was delivered in a colletive meeting to all teachers in the school. The control group schools, randomly selected, will simply receive our intervention later, in spring 2018.

Experimental Design Details

Randomization Method

Randomization done in office by a computer, by a researcher external to the project staff (Giovanni Abbiati - IRVAPP, [email protected]

Randomization Unit

schools (whithin blocks)

Was the treatment clustered?

Yes

Experiment Characteristics

Sample size: planned number of clusters

198 schools whithin 8 blocks

Sample size: planned number of observations

around 2.200 teachers teaching Math or Italian to 7th graders around 24.000 students for each grade (6th, 7th, 8th)

Sample size (or number of clusters) by treatment arms

50 schools individual treatment
50 schools collective treatment
98 schools control

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

* Please, note: conservative estimates Teachers' outcomes - around 0,156 express in terms of effect size - both treatments considered together vs control Teachers' outcomes - around 0,193 express in terms of effect size - each treatment considered separately vs control Students' outcomes - around 0,084 express in terms of effect size - both treatments considered together vs control Students' outcomes - around 0,104 express in terms of effect size - each treatment considered separately vs control

Supporting Documents and Materials

IRB