Field
Trial Title
|
Before
Behavioral Evidence for Non-Cognitive Skills Development
|
After
The Impact of Formative Assessment of Behavior-Based Socioemotional Skills on Students' Outcomes in the Short and Long Run
|
Field
Abstract
|
Before
It is widely recognized that social and personal skills (i.e., perseverance, motivation, teamwork, etc.) are highly predictive of life achievements and long-term well-being, such as lower levels of school dropout, physical and mental health issues, and conflict. It is also well established that a comprehensive integration of these non-cognitive skills in the educational curriculum is essential to make lifetime progress.
The objective of this trial is to test the causal impact of integrating formative assessment on social and emotional skills on academic performance and non-cognitive skill development through a novel methodology called Pentabilities. The Pentabilities methodology enables teachers to observe and give feedback on a concrete set of observable, measurable, and identifiable behaviors in active classroom environments that characterize non-cognitive skills. The teachers are given 5-6 months to implement the proposed methodology. The trial involves 40 Catalan secondary schools that mainly serve at-risk populations.
|
After
It is widely recognized that social and personal skills (i.e., perseverance, motivation, teamwork, etc.) are highly predictive of life achievements and long-term well-being, such as lower levels of school dropout, physical and mental health issues, and conflict. It is also well established that a comprehensive integration of these non-cognitive skills in the educational curriculum is essential to make lifetime progress.
The objective of this trial is to test the causal impact of training and mentoring teachers to integrate formative assessment of socioemotional skills in the classroom with the help of digital tools on students’ academic and non-cognitive outcomes. Formative assessment by the trained teachers involves the observation, recording, and provision of feedback on a specific set of behaviors, the so-called Pentabilities, that characterize socioemotional skills in active classroom environments. The teachers are given 5-6 months to implement the intervention. The trial involves 40 Catalan secondary schools that mainly serve at-risk populations.
|
Field
Last Published
|
Before
January 12, 2023 05:35 PM
|
After
February 13, 2023 04:24 PM
|
Field
Intervention (Public)
|
Before
|
After
The intervention consists of training and mentoring teachers to integrate formative assessment of socioemotional skills into the existing curriculum with the help of digital tools.
|
Field
Primary Outcomes (End Points)
|
Before
1) Observational data on behavior-based socioemotional skills
2) Survey-based measures of socioemotional skills (self-reported)
3) Survey measures of socioemotional skills (observer-reported)
4) Academic performance and attendance
|
After
1) Observational data on behavior-based socioemotional skills
2) Academic performance
3) Attendance
|
Field
Primary Outcomes (Explanation)
|
Before
The tools that are used to measure the outcomes indicated in items 1-3 are mostly novel instruments that have not been used before with this specific population and in a translated language (Catalan/Spanish). Therefore, we will first study the distribution of these variables in the baseline dataset (before the implementation period starts in January 2023 and before we collect any treatment data) to gauge how to best construct the socioemotional skill outcomes.
Academic performance will be measured as the average of the subject grades (GPA) that appears in school report cards.
Attendance will be measured as the number of days missed during the school year 2022-2023.
|
After
|
Field
Planned Number of Clusters
|
Before
Overall clusters of 85 grades, in 40 schools, with a total of 3,230 pupils.
|
After
Overall clusters of 85 grades, in 40 schools, with a total of potential study sample of 3,648 students and 98 teachers.
|
Field
Planned Number of Observations
|
Before
Typical class size of 20 students with 1.9 classes per grade per 85 grades= 3,230 pupils. We have 85 grades, 1.9 classes in each grade, and a total of 40 schools.
|
After
Typical class size of 20 students with 1.9 classes per grade per 85 grades amounts to 3,230 pupils. We have 85 grades, 1.9 classes in each grade, and a total of 40 schools in the study sample.
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
It is a Treatment vs. Control, single arm study. Treated/control pupils= 1,672/1,558 treated/control grades=44/41.
|
After
It is a Treatment vs. Control, single arm study. Treated/control pupils= 1,844/1,839 treated/control grades=44/41.
|
Field
Intervention (Hidden)
|
Before
|
After
The teachers randomized into treatment receive comprehensive training and mentoring on how to implement formative assessment of socioemotional skills in a classroom setting. They complete an initial 8-hours workshop on social and emotional skills (4-hour remote training using self-paced videos, and 4-hours in-person training with case studies and hands-on group activities), where teachers learn how to observe, record, and provide feedback on socioemotional skills to their students, and familiarize themselves with the digital tools used to record and track the behavioral evidence. Treated teachers also have monthly one-on-one meetings with a mentor to discuss potential challenges and solutions on how to improve on integrating socioemotional skill development in the classroom dynamics. In addition, there are two group sessions where ten to fifteen teachers get together to discuss two topics. First, how to use the collected evidence to provide feedback (2 months after the first training), and second, a focus group to discuss challenges and solutions that teachers have faced (4 months after the first training).
The training and mentoring sessions guide teachers to adapt the classroom dynamics and plan the sessions to be able to observe and collect evidence on the Pentabilities behaviors during active classroom environments. Teachers plan a moment of the session where they focus on a subset of the students, collecting and rating from 1-to-5 the behaviors they are able to observe. They also create moments where students evaluate themselves and some of their peers (co-evaluations). These 360º evaluations are collected on a recurrent basis for all students. The data collection is supported by the mobile app. All of the evidence is automatically aggregated across time in individualized reports, which are available in a user-friendly format on the web platform. With growing data, teachers are able to organize feedback sessions using the student reports, where strengths and weaknesses are emphasized, and evidence from different sources (self, peer, and teacher evaluations) can be compared. Feedback sessions can be done through group reflections---guided by specific issues---or through one-on-one sessions with the teacher. The result of these sessions is a personalized plan of action to further develop the socioemotional skills.
The intervention will be implemented from January to June 2023.
|