Back to History

Fields Changed

Registration

Field Before After
Trial Title Behavioral Evidence for Non-Cognitive Skills Development The Impact of Formative Assessment of Behavior-Based Socioemotional Skills on Students' Outcomes in the Short and Long Run
Abstract It is widely recognized that social and personal skills (i.e., perseverance, motivation, teamwork, etc.) are highly predictive of life achievements and long-term well-being, such as lower levels of school dropout, physical and mental health issues, and conflict. It is also well established that a comprehensive integration of these non-cognitive skills in the educational curriculum is essential to make lifetime progress. The objective of this trial is to test the causal impact of integrating formative assessment on social and emotional skills on academic performance and non-cognitive skill development through a novel methodology called Pentabilities. The Pentabilities methodology enables teachers to observe and give feedback on a concrete set of observable, measurable, and identifiable behaviors in active classroom environments that characterize non-cognitive skills. The teachers are given 5-6 months to implement the proposed methodology. The trial involves 40 Catalan secondary schools that mainly serve at-risk populations. It is widely recognized that social and personal skills (i.e., perseverance, motivation, teamwork, etc.) are highly predictive of life achievements and long-term well-being, such as lower levels of school dropout, physical and mental health issues, and conflict. It is also well established that a comprehensive integration of these non-cognitive skills in the educational curriculum is essential to make lifetime progress. The objective of this trial is to test the causal impact of training and mentoring teachers to integrate formative assessment of socioemotional skills in the classroom with the help of digital tools on students’ academic and non-cognitive outcomes. Formative assessment by the trained teachers involves the observation, recording, and provision of feedback on a specific set of behaviors, the so-called Pentabilities, that characterize socioemotional skills in active classroom environments. The teachers are given 5-6 months to implement the intervention. The trial involves 40 Catalan secondary schools that mainly serve at-risk populations.
Last Published January 12, 2023 05:35 PM February 13, 2023 04:24 PM
Intervention (Public) The intervention consists of training and mentoring teachers to integrate formative assessment of socioemotional skills into the existing curriculum with the help of digital tools.
Primary Outcomes (End Points) 1) Observational data on behavior-based socioemotional skills 2) Survey-based measures of socioemotional skills (self-reported) 3) Survey measures of socioemotional skills (observer-reported) 4) Academic performance and attendance 1) Observational data on behavior-based socioemotional skills 2) Academic performance 3) Attendance
Primary Outcomes (Explanation) The tools that are used to measure the outcomes indicated in items 1-3 are mostly novel instruments that have not been used before with this specific population and in a translated language (Catalan/Spanish). Therefore, we will first study the distribution of these variables in the baseline dataset (before the implementation period starts in January 2023 and before we collect any treatment data) to gauge how to best construct the socioemotional skill outcomes. Academic performance will be measured as the average of the subject grades (GPA) that appears in school report cards. Attendance will be measured as the number of days missed during the school year 2022-2023.
Planned Number of Clusters Overall clusters of 85 grades, in 40 schools, with a total of 3,230 pupils. Overall clusters of 85 grades, in 40 schools, with a total of potential study sample of 3,648 students and 98 teachers.
Planned Number of Observations Typical class size of 20 students with 1.9 classes per grade per 85 grades= 3,230 pupils. We have 85 grades, 1.9 classes in each grade, and a total of 40 schools. Typical class size of 20 students with 1.9 classes per grade per 85 grades amounts to 3,230 pupils. We have 85 grades, 1.9 classes in each grade, and a total of 40 schools in the study sample.
Sample size (or number of clusters) by treatment arms It is a Treatment vs. Control, single arm study. Treated/control pupils= 1,672/1,558 treated/control grades=44/41. It is a Treatment vs. Control, single arm study. Treated/control pupils= 1,844/1,839 treated/control grades=44/41.
Intervention (Hidden) The teachers randomized into treatment receive comprehensive training and mentoring on how to implement formative assessment of socioemotional skills in a classroom setting. They complete an initial 8-hours workshop on social and emotional skills (4-hour remote training using self-paced videos, and 4-hours in-person training with case studies and hands-on group activities), where teachers learn how to observe, record, and provide feedback on socioemotional skills to their students, and familiarize themselves with the digital tools used to record and track the behavioral evidence. Treated teachers also have monthly one-on-one meetings with a mentor to discuss potential challenges and solutions on how to improve on integrating socioemotional skill development in the classroom dynamics. In addition, there are two group sessions where ten to fifteen teachers get together to discuss two topics. First, how to use the collected evidence to provide feedback (2 months after the first training), and second, a focus group to discuss challenges and solutions that teachers have faced (4 months after the first training). The training and mentoring sessions guide teachers to adapt the classroom dynamics and plan the sessions to be able to observe and collect evidence on the Pentabilities behaviors during active classroom environments. Teachers plan a moment of the session where they focus on a subset of the students, collecting and rating from 1-to-5 the behaviors they are able to observe. They also create moments where students evaluate themselves and some of their peers (co-evaluations). These 360º evaluations are collected on a recurrent basis for all students. The data collection is supported by the mobile app. All of the evidence is automatically aggregated across time in individualized reports, which are available in a user-friendly format on the web platform. With growing data, teachers are able to organize feedback sessions using the student reports, where strengths and weaknesses are emphasized, and evidence from different sources (self, peer, and teacher evaluations) can be compared. Feedback sessions can be done through group reflections---guided by specific issues---or through one-on-one sessions with the teacher. The result of these sessions is a personalized plan of action to further develop the socioemotional skills. The intervention will be implemented from January to June 2023.
Back to top

Partners

Field Before After
Partner Name Ministerio de Inclusión, Seguridad Social y Migraciones
Partner Type government
Partner Website (URL) https://www.inclusion.gob.es/web/guest/home
Public Yes
Back to top
Field Before After
Partner Name Fundació Bofill
Partner Type ngo
Partner Website (URL) https://fundaciobofill.cat/
Public Yes
Back to top