Teachers or School Leaders? Artificial Intelligence and Teaching Complementarities

This project involves a randomized control trial conducted across 91 eligible schools in the Dominican Republic. Teachers and school leaders (i.e., those responsible for the academic management of the institution) will be given access to ShaIA, an AI-powered platform designed to support educators before, during, and after class.

ShaIA is an AI-driven pedagogical ecosystem that provides personalized guidance to teachers. The platform is designed to support mathematics instruction by assisting teachers in planning and implementing lessons aligned with curricular standards and students’ diverse learning needs. Before each class, teachers input information such as learning objectives, class size, duration, available resources, and the type of activity planned. Based on this input and prior evidence, ShaIA generates detailed instructional guidance to help teachers implement effective lessons. During class, the platform collects data on various teaching and classroom attributes. These data is then used to generate individualized, targeted feedback for each teacher, tailored to their specific teaching style and classroom dynamics.

2025-09-01

2026-06-30

Student learning captures by test scores

To quantify the effects on student learning outcomes, we will administer two standardized exams to all students, regardless of whether they attend a treatment or control school. A first exam will evaluate students in math knowledge and the second in Spanish language. Each assessment will consist of a 45-minute multiple choice test conducted via a digital platform. The first round of testing will take place before the intervention begins, and the second will be administered at the end of the academic year. In addition, we will collect data on students’ demographic and socio-economic characteristics.

Teacher and school leaders engagement with the tool

For ShaIA to be effective, teachers and school leaders must adapt their practices by integrating the platform into their class preparation and their academic duties. This may influence how they plan lessons, potentially reducing preparation time and increasing engagement with the tool. Furthermore, teachers who adopt the technology may employ new pedagogical strategies that are more effective in improving student learning. To evaluate the effects on teaching and administration practices, we will use three different instruments:

• Teacher surveys, which will collect information on how teachers prepare their classes, including the tools they use and the amount of time spent on preparation.

• School Leader Surveys, which will gather data on how leaders monitor teaching practices, communicate with students' parents, and provide support and resources to teachers.

• In-class observations, conducted by trained assessors who will attend lessons and complete a structured observation form. These forms will document time allocation, use of pedagogical tools, and overall classroom practices.

To evaluate the impact of ShaIA, and recognizing that its effectiveness may differ depending on whether access is granted to teachers or administrators, we will randomly assign the 91 schools into one of three groups:

1) Treatment 1: 13 schools where both all teachers and school leaders will receive access to the AI tools.
2) Treatment 2: 12 schools where all teachers and only one school administrator will receive access.
3) Control: 66 schools that will continue with their existing practices and will not receive access to the AI tools.

This design will allow us to estimate not only the overall effect of ShaIA on student learning, but also to explore how different implementation strategies influence its impact.

Our study population includes students, teachers, and school leaders from schools participating in a strategic alliance between the Ministry of Education of the Dominican Republic (MINERD) and Instituto 512, a privately funded organization that supports teachers in the Dominican Republic by providing resources to enhance their instructional practices. Since 2021, this partnership has implemented a teaching strategy called the Modelo FARO in 187 selected schools. The FARO model provides schools with evidence-based resources to improve student learning through diagnostics, professional guidance, and training for teachers and administrators.

This evaluation focuses specifically on primary school students and teachers in grades 4, 5, and 6. Due to resource constraints, we selected a subset of 91 schools from the original 187. These schools were chosen based on two criteria: 1) they had students in the target grades; and 2) they employed no more than 14 primary school teachers. This selection allows us to provide nearly all teachers at each school with access to the ShaIA platform. By focusing on schools where full teacher coverage is feasible, we avoid within-school sampling bias and remain within our resource limits. The resulting sample includes approximately 15,000 students, 600 teachers, and 750 school administrators.

Heterogeneity:
In addition to modifying teaching practices, ShaIA may have differentiated effects if the platform enables teachers to better tailor instruction to specific types of students. For example, learning gains might vary by gender or socio-economic status if the platform helps teachers adapt their methods to accommodate different learning needs. To assess this, we will estimate heterogeneous treatment effects using the test score data by gender and socio-economic background. These characteristics will be gathered through a student survey. Gender will be computed asking directly to the student, whereas socio-economic status will be measured using a wealth index constructed via principal component analysis, based on students’ reported access to household assets—a commonly used method in developing country settings.

Not available

Randomization was performed in a computer using an algorithm that randomly assigns a value to each school. Using this vector, schools were sorted and randomly assigned to each of the three treatment arms.

The randomization units are schools that 1) had students in the target grades; and 2) employed no more than 14 primary school teachers.

Yes

The evaluation includes a total of 91 eligible schools. Out of these, Treatment 1 includes 13 schools, Treatment 2 includes 12 schools, and 66 schools were assigned to the control group.

Based on data for the 2024-2025 school year, this design includes, approximately, 15,346 students, 750 school leaders, and 590 teachers. Final sample sizes might vary slightly as the intervention will take place during the 2025-2026 school year. Due to budget constraints, a random subsample of students will be gathered in 25 schools of the control group. All treatment schools will be included in the data collection.

Based on data for the 2024-2025 school year, this design includes, approximately, 4,000 students, 178 teachers, and 199 school leaders, in the treatment groups. The control group includes around 12,000 students, 400 teachers, and 500 school leaders. Final sample sizes might vary slightly as the intervention will take place during the 2025-2026 school year. Due to budget constraints, a random subsample of students will be gathered in 25 schools of the control group. All treatment schools will be included in the data collections.

This sample sizes are enough to properly identify treatment effects. Assuming an average of 25 students per teacher, with 100 teachers in the treatment group and 400 in the control group, the study is powered to detect a minimum effect size of 0.09 standard deviations. This is below the average effect size typically observed in randomized controlled trials in education (Evans and Yuan 2022), suggesting our sample size is sufficient to detect meaningful impacts, even with clustering at the teacher level