The Impact of Formative Assessment of Behavior-Based Socioemotional Skills on Students' Outcomes in the Short and Long Run

Last registered on July 05, 2023

Pre-Trial

Trial Information

General Information

Title
The Impact of Formative Assessment of Behavior-Based Socioemotional Skills on Students' Outcomes in the Short and Long Run
RCT ID
AEARCTR-0010447
Initial registration date
November 28, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
November 30, 2022, 4:41 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
July 05, 2023, 5:54 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Institute of Political Economy and Governance

Other Primary Investigator(s)

PI Affiliation
University of Geneva, IEE/GSEM
PI Affiliation
Universitat Autònoma de Barcelona
PI Affiliation
Stockholm University IIES

Additional Trial Information

Status
On going
Start date
2022-11-07
End date
2024-03-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
It is widely recognized that social and personal skills (i.e., perseverance, motivation, teamwork, etc.) are highly predictive of life achievements and long-term well-being, such as lower levels of school dropout, physical and mental health issues, and conflict. It is also well established that a comprehensive integration of these non-cognitive skills in the educational curriculum is essential to make lifetime progress.

The objective of this trial is to test the causal impact of training and mentoring teachers to integrate formative assessment of socioemotional skills in the classroom with the help of digital tools on students’ academic and non-cognitive outcomes. Formative assessment by the trained teachers involves the observation, recording, and provision of feedback on a specific set of behaviors, the so-called Pentabilities, that characterize socioemotional skills in active classroom environments. The teachers are given 5-6 months to implement the intervention. The trial involves 40 Catalan secondary schools that mainly serve at-risk populations.
External Link(s)

Registration Citation

Citation
Calsamiglia, Caterina et al. 2023. "The Impact of Formative Assessment of Behavior-Based Socioemotional Skills on Students' Outcomes in the Short and Long Run ." AEA RCT Registry. July 05. https://doi.org/10.1257/rct.10447-5.0
Experimental Details

Interventions

Intervention(s)
The intervention consists of training and mentoring teachers to integrate formative assessment of socioemotional skills into the existing curriculum with the help of digital tools.
Intervention Start Date
2022-11-28
Intervention End Date
2023-06-30

Primary Outcomes

Primary Outcomes (end points)
1) Behavior-based observational measures of socioemotional skills
2) Measure of awareness about socioemotional skills
2) Academic performance
3) Attendance
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The study is a clustered randomized controlled trial, where the unit of randomization is the grade level (i.e., 1st, 2nd, 3rd, and 4th ESO). The recruited sample is based on teachers voluntarily participating in the project with the support of their schools. In total, there are 85 grade levels across 40 schools in the Catalunya region recruited for the study. We randomly assigned 41 grade levels to control and 44 grades levels to the treatment group.
Experimental Design Details
Randomization Method
Randomization was performed using a computer code.
Randomization Unit
The treatment is assigned at the grade level.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
Overall clusters of 85 grades, in 40 schools, with a total of potential study sample of 3,648 students and 98 teachers.
Sample size: planned number of observations
Typical class size of 20 students with 1.9 classes per grade per 85 grades amounts to 3,230 pupils. We have 85 grades, 1.9 classes in each grade, and a total of 40 schools in the study sample.
Sample size (or number of clusters) by treatment arms
It is a Treatment vs. Control, single arm study. Treated/control pupils= 1,844/1,839 treated/control grades=44/41.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We can detect a minimum effect of 22.75% of a standard deviation (.2275sigma) with 80% power with 85 clusters. This is under the assumption of an Intra Cluster Correlation of .1 (as per data from pentabilities), mean pentabilities score 4, standard deviation of .9. This is an increase in the Pentabilities score of about .2. These are very conservative scenarios, as we are not considering multiple measurements.
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
COMITÉ ÉTICO VINCULADO A LOS ITINERARIOS DE INCLUSIÓN SOCIAL
IRB Approval Date
2022-11-10
IRB Approval Number
N/A
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials