The Impact of Formative Assessment of Behavior-Based Socioemotional Skills on Students' Outcomes in the Short and Long Run

Last registered on July 05, 2023

Pre-Trial

General Information

Title

RCT ID

AEARCTR-0010447

Initial registration date

November 28, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

November 30, 2022, 4:41 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated

July 05, 2023, 5:54 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country

Spain

Region

Catalunya

Primary Investigator

Name

Caterina Calsamiglia

Affiliation

Institute of Political Economy and Governance

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Giacomo De Giorgi

PI Affiliation

University of Geneva, IEE/GSEM

Contact Investigator

PI Name

Ece Yagman

PI Affiliation

Universitat Autònoma de Barcelona

Contact Investigator

PI Name

Laia Navarro-Sola

PI Affiliation

Stockholm University IIES

Contact Investigator

Additional Trial Information

Status

On going

Start date

2022-11-07

End date

2024-03-31

Keywords

Education

Additional Keywords

JEL code(s)

Secondary IDs

Prior work

This trial does not extend or rely on any prior RCTs.

Abstract

It is widely recognized that social and personal skills (i.e., perseverance, motivation, teamwork, etc.) are highly predictive of life achievements and long-term well-being, such as lower levels of school dropout, physical and mental health issues, and conflict. It is also well established that a comprehensive integration of these non-cognitive skills in the educational curriculum is essential to make lifetime progress.

The objective of this trial is to test the causal impact of training and mentoring teachers to integrate formative assessment of socioemotional skills in the classroom with the help of digital tools on students’ academic and non-cognitive outcomes. Formative assessment by the trained teachers involves the observation, recording, and provision of feedback on a specific set of behaviors, the so-called Pentabilities, that characterize socioemotional skills in active classroom environments. The teachers are given 5-6 months to implement the intervention. The trial involves 40 Catalan secondary schools that mainly serve at-risk populations.

External Link(s)

Registration Citation

Citation

Calsamiglia, Caterina et al. 2023. "The Impact of Formative Assessment of Behavior-Based Socioemotional Skills on Students' Outcomes in the Short and Long Run ." AEA RCT Registry. July 05. https://doi.org/10.1257/rct.10447-5.0

Sponsors & Partners

Partner

Name

Ministerio de Inclusión, Seguridad Social y Migraciones

Type

government

URL

https://www.inclusion.gob.es/web/guest/home

Name

Fundació Bofill

Type

ngo

URL

https://fundaciobofill.cat/

Experimental Details

Interventions

Intervention(s)

The intervention consists of training and mentoring teachers to integrate formative assessment of socioemotional skills into the existing curriculum with the help of digital tools.

Intervention Start Date

2022-11-28

Intervention End Date

2023-06-30

Primary Outcomes

Primary Outcomes (end points)

1) Behavior-based observational measures of socioemotional skills
2) Measure of awareness about socioemotional skills
2) Academic performance
3) Attendance

Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)

Secondary Outcomes (explanation)

Experimental Design

The study is a clustered randomized controlled trial, where the unit of randomization is the grade level (i.e., 1st, 2nd, 3rd, and 4th ESO). The recruited sample is based on teachers voluntarily participating in the project with the support of their schools. In total, there are 85 grade levels across 40 schools in the Catalunya region recruited for the study. We randomly assigned 41 grade levels to control and 44 grades levels to the treatment group.

Experimental Design Details

Randomization Method

Randomization was performed using a computer code.

Randomization Unit

The treatment is assigned at the grade level.

Was the treatment clustered?

Yes

Experiment Characteristics

Sample size: planned number of clusters

Overall clusters of 85 grades, in 40 schools, with a total of potential study sample of 3,648 students and 98 teachers.

Sample size: planned number of observations

Typical class size of 20 students with 1.9 classes per grade per 85 grades amounts to 3,230 pupils. We have 85 grades, 1.9 classes in each grade, and a total of 40 schools in the study sample.

Sample size (or number of clusters) by treatment arms

It is a Treatment vs. Control, single arm study. Treated/control pupils= 1,844/1,839 treated/control grades=44/41.

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

We can detect a minimum effect of 22.75% of a standard deviation (.2275sigma) with 80% power with 85 clusters. This is under the assumption of an Intra Cluster Correlation of .1 (as per data from pentabilities), mean pentabilities score 4, standard deviation of .9. This is an increase in the Pentabilities score of about .2. These are very conservative scenarios, as we are not considering multiple measurements.

Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

IRB