Behavioral Evidence for Non-Cognitive Skills Development

Last registered on January 12, 2023

Pre-Trial

Trial Information

General Information

Title
Behavioral Evidence for Non-Cognitive Skills Development
RCT ID
AEARCTR-0010447
Initial registration date
November 28, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
November 30, 2022, 4:41 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
January 12, 2023, 5:35 PM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Institute of Political Economy and Governance

Other Primary Investigator(s)

PI Affiliation
University of Geneva, IEE/GSEM
PI Affiliation
Universitat Autònoma de Barcelona
PI Affiliation
Stockholm University IIES

Additional Trial Information

Status
In development
Start date
2022-11-07
End date
2024-03-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
It is widely recognized that social and personal skills (i.e., perseverance, motivation, teamwork, etc.) are highly predictive of life achievements and long-term well-being, such as lower levels of school dropout, physical and mental health issues, and conflict. It is also well established that a comprehensive integration of these non-cognitive skills in the educational curriculum is essential to make lifetime progress.

The objective of this trial is to test the causal impact of integrating formative assessment on social and emotional skills on academic performance and non-cognitive skill development through a novel methodology called Pentabilities. The Pentabilities methodology enables teachers to observe and give feedback on a concrete set of observable, measurable, and identifiable behaviors in active classroom environments that characterize non-cognitive skills. The teachers are given 5-6 months to implement the proposed methodology. The trial involves 40 Catalan secondary schools that mainly serve at-risk populations.
External Link(s)

Registration Citation

Citation
Calsamiglia, Caterina et al. 2023. "Behavioral Evidence for Non-Cognitive Skills Development ." AEA RCT Registry. January 12. https://doi.org/10.1257/rct.10447-1.1
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2022-11-28
Intervention End Date
2023-06-30

Primary Outcomes

Primary Outcomes (end points)
1) Observational data on behavior-based socioemotional skills
2) Survey-based measures of socioemotional skills (self-reported)
3) Survey measures of socioemotional skills (observer-reported)
4) Academic performance and attendance
Primary Outcomes (explanation)
The tools that are used to measure the outcomes indicated in items 1-3 are mostly novel instruments that have not been used before with this specific population and in a translated language (Catalan/Spanish). Therefore, we will first study the distribution of these variables in the baseline dataset (before the implementation period starts in January 2023 and before we collect any treatment data) to gauge how to best construct the socioemotional skill outcomes.

Academic performance will be measured as the average of the subject grades (GPA) that appears in school report cards.

Attendance will be measured as the number of days missed during the school year 2022-2023.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The study is a clustered randomized controlled trial, where the unit of randomization is the grade level (i.e., 1st, 2nd, 3rd, and 4th ESO). The recruited sample is based on teachers voluntarily participating in the project with the support of their schools. In total, there are 85 grade levels across 40 schools in the Catalunya region recruited for the study. We randomly assigned 41 grade levels to control and 44 grades levels to the treatment group.
Experimental Design Details
Not available
Randomization Method
Randomization was performed using a computer code.
Randomization Unit
The treatment is assigned at the grade level.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
Overall clusters of 85 grades, in 40 schools, with a total of 3,230 pupils.
Sample size: planned number of observations
Typical class size of 20 students with 1.9 classes per grade per 85 grades= 3,230 pupils. We have 85 grades, 1.9 classes in each grade, and a total of 40 schools.
Sample size (or number of clusters) by treatment arms
It is a Treatment vs. Control, single arm study. Treated/control pupils= 1,672/1,558 treated/control grades=44/41.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We can detect a minimum effect of 22.75% of a standard deviation (.2275sigma) with 80% power with 85 clusters. This is under the assumption of an Intra Cluster Correlation of .1 (as per data from pentabilities), mean pentabilities score 4, standard deviation of .9. This is an increase in the Pentabilities score of about .2. These are very conservative scenarios, as we are not considering multiple measurements.
IRB

Institutional Review Boards (IRBs)

IRB Name
COMITÉ ÉTICO VINCULADO A LOS ITINERARIOS DE INCLUSIÓN SOCIAL
IRB Approval Date
2022-11-10
IRB Approval Number
N/A