The Impact of Formative Assessment of Behavior-Based Socioemotional Skills on Students' Outcomes in the Short and Long Run

Last registered on November 30, 2022

View Trial History

Pre-Trial

Trial Information

General Information

Title

Behavioral Evidence for Non-Cognitive Skills Development

RCT ID

AEARCTR-0010447

Initial registration date

November 28, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

November 30, 2022, 4:41 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country

Spain

Region

Catalunya

Primary Investigator

Name

Caterina Calsamiglia

Affiliation

Institute of Political Economy and Governance

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Laia Navarro-Solà

PI Affiliation

International Economics Studies at Stockholm University

Contact Investigator

PI Name

Giacomo De Giorgi

PI Affiliation

University of Geneva, IEE/GSEM

Contact Investigator

PI Name

Ece Yagman

PI Affiliation

Universitat Autònoma de Barcelona

Contact Investigator

Additional Trial Information

Status

In development

Start date

2022-11-07

End date

2024-03-31

Keywords

Education

Additional Keywords

JEL code(s)

Secondary IDs

Prior work

This trial does not extend or rely on any prior RCTs.

Abstract

It is widely recognized that social and personal skills (i.e., perseverance, motivation, teamwork, etc.) are highly predictive of life achievements and long-term well-being, such as lower levels of school dropout, physical and mental health issues, and conflict. It is also well established that a comprehensive integration of these non-cognitive skills in the educational curriculum is essential to make lifetime progress.

The objective of this trial is to test the causal impact of integrating formative assessment on social and emotional skills on academic performance and non-cognitive skill development through a novel methodology called Pentabilities. The Pentabilities methodology enables teachers to observe and give feedback on a concrete set of observable, measurable, and identifiable behaviors in active classroom environments that characterize non-cognitive skills. The teachers are given 5-6 months to implement the proposed methodology. The trial involves 40 Catalan secondary schools that mainly serve at-risk populations.

External Link(s)

Registration Citation

Citation

Calsamiglia, Caterina et al. 2022. "Behavioral Evidence for Non-Cognitive Skills Development ." AEA RCT Registry. November 30. https://doi.org/10.1257/rct.10447-1.0

Sponsors & Partners

Experimental Details

Interventions

Intervention(s)

Intervention Start Date

2022-11-28

Intervention End Date

2023-06-30

Primary Outcomes

Primary Outcomes (end points)

1) Observational data on behavior-based socioemotional skills
2) Survey-based measures of socioemotional skills (self-reported)
3) Survey measures of socioemotional skills (observer-reported)
4) Academic performance and attendance

Primary Outcomes (explanation)

The tools that are used to measure the outcomes indicated in items 1-3 are mostly novel instruments that have not been used before with this specific population and in a translated language (Catalan/Spanish). Therefore, we will first study the distribution of these variables in the baseline dataset (before the implementation period starts in January 2023 and before we collect any treatment data) to gauge how to best construct the socioemotional skill outcomes.

Academic performance will be measured as the average of the subject grades (GPA) that appears in school report cards.

Attendance will be measured as the number of days missed during the school year 2022-2023.

Secondary Outcomes

Secondary Outcomes (end points)

Secondary Outcomes (explanation)

Experimental Design

The study is a clustered randomized controlled trial, where the unit of randomization is the grade level (i.e., 1st, 2nd, 3rd, and 4th ESO). The recruited sample is based on teachers voluntarily participating in the project with the support of their schools. In total, there are 85 grade levels across 40 schools in the Catalunya region recruited for the study. We randomly assigned 41 grade levels to control and 44 grades levels to the treatment group.

Experimental Design Details

Randomization Method

Randomization was performed using a computer code.

Randomization Unit

The treatment is assigned at the grade level.

Was the treatment clustered?

Yes

Experiment Characteristics

Sample size: planned number of clusters

Overall clusters of 85 grades, in 40 schools, with a total of 3,230 pupils.

Sample size: planned number of observations

Typical class size of 20 students with 1.9 classes per grade per 85 grades= 3,230 pupils. We have 85 grades, 1.9 classes in each grade, and a total of 40 schools.

Sample size (or number of clusters) by treatment arms

It is a Treatment vs. Control, single arm study. Treated/control pupils= 1,672/1,558 treated/control grades=44/41.

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

We can detect a minimum effect of 22.75% of a standard deviation (.2275sigma) with 80% power with 85 clusters. This is under the assumption of an Intra Cluster Correlation of .1 (as per data from pentabilities), mean pentabilities score 4, standard deviation of .9. This is an increase in the Pentabilities score of about .2. These are very conservative scenarios, as we are not considering multiple measurements.

Supporting Documents and Materials

IRB