Back to History Current Version

The Impact of Formative Assessment of Behavior-Based Socioemotional Skills on Students' Outcomes in the Short and Long Run

Last registered on November 30, 2022
View Trial History

Pre-Trial

Trial Information

General Information

Title
Behavioral Evidence for Non-Cognitive Skills Development
RCT ID
AEARCTR-0010447
Initial registration date
November 28, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
November 30, 2022, 4:41 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
Spain
Region
Catalunya

Primary Investigator

Name
Caterina Calsamiglia
Affiliation
Institute of Political Economy and Governance
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Laia Navarro-Solà
PI Affiliation
International Economics Studies at Stockholm University
Contact Investigator
PI Name
Giacomo De Giorgi
PI Affiliation
University of Geneva, IEE/GSEM
Contact Investigator
PI Name
Ece Yagman
PI Affiliation
Universitat Autònoma de Barcelona
Contact Investigator

Additional Trial Information

Status
In development
Start date
2022-11-07
End date
2024-03-31
Keywords
Education
Additional Keywords
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
It is widely recognized that social and personal skills (i.e., perseverance, motivation, teamwork, etc.) are highly predictive of life achievements and long-term well-being, such as lower levels of school dropout, physical and mental health issues, and conflict. It is also well established that a comprehensive integration of these non-cognitive skills in the educational curriculum is essential to make lifetime progress.

The objective of this trial is to test the causal impact of integrating formative assessment on social and emotional skills on academic performance and non-cognitive skill development through a novel methodology called Pentabilities. The Pentabilities methodology enables teachers to observe and give feedback on a concrete set of observable, measurable, and identifiable behaviors in active classroom environments that characterize non-cognitive skills. The teachers are given 5-6 months to implement the proposed methodology. The trial involves 40 Catalan secondary schools that mainly serve at-risk populations.
External Link(s)

Registration Citation

Citation
Calsamiglia, Caterina et al. 2022. "Behavioral Evidence for Non-Cognitive Skills Development ." AEA RCT Registry. November 30. https://doi.org/10.1257/rct.10447-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2022-11-28
Intervention End Date
2023-06-30

Primary Outcomes

Primary Outcomes (end points)
1) Observational data on behavior-based socioemotional skills
2) Survey-based measures of socioemotional skills (self-reported)
3) Survey measures of socioemotional skills (observer-reported)
4) Academic performance and attendance
Primary Outcomes (explanation)
The tools that are used to measure the outcomes indicated in items 1-3 are mostly novel instruments that have not been used before with this specific population and in a translated language (Catalan/Spanish). Therefore, we will first study the distribution of these variables in the baseline dataset (before the implementation period starts in January 2023 and before we collect any treatment data) to gauge how to best construct the socioemotional skill outcomes.

Academic performance will be measured as the average of the subject grades (GPA) that appears in school report cards.

Attendance will be measured as the number of days missed during the school year 2022-2023.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The study is a clustered randomized controlled trial, where the unit of randomization is the grade level (i.e., 1st, 2nd, 3rd, and 4th ESO). The recruited sample is based on teachers voluntarily participating in the project with the support of their schools. In total, there are 85 grade levels across 40 schools in the Catalunya region recruited for the study. We randomly assigned 41 grade levels to control and 44 grades levels to the treatment group.
Experimental Design Details
Not available
Randomization Method
Randomization was performed using a computer code.
Randomization Unit
The treatment is assigned at the grade level.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
Overall clusters of 85 grades, in 40 schools, with a total of 3,230 pupils.
Sample size: planned number of observations
Typical class size of 20 students with 1.9 classes per grade per 85 grades= 3,230 pupils. We have 85 grades, 1.9 classes in each grade, and a total of 40 schools.
Sample size (or number of clusters) by treatment arms
It is a Treatment vs. Control, single arm study. Treated/control pupils= 1,672/1,558 treated/control grades=44/41.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We can detect a minimum effect of 22.75% of a standard deviation (.2275sigma) with 80% power with 85 clusters. This is under the assumption of an Intra Cluster Correlation of .1 (as per data from pentabilities), mean pentabilities score 4, standard deviation of .9. This is an increase in the Pentabilities score of about .2. These are very conservative scenarios, as we are not considering multiple measurements.
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
COMITÉ ÉTICO VINCULADO A LOS ITINERARIOS DE INCLUSIÓN SOCIAL
IRB Approval Date
2022-11-10
IRB Approval Number
N/A
Analysis Plan