Maximizing the Impact of Analyzing Previous Educational Results: Communication and Monitoring for School Improvement

Last registered on December 20, 2024

Pre-Trial

Trial Information

General Information

Title
Maximizing the Impact of Analyzing Previous Educational Results: Communication and Monitoring for School Improvement
RCT ID
AEARCTR-0014955
Initial registration date
December 11, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
December 20, 2024, 12:20 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Independent

Other Primary Investigator(s)

PI Affiliation
Universidad de los Andes

Additional Trial Information

Status
In development
Start date
2024-12-02
End date
2025-06-30
Secondary IDs
Prior work
This trial is based on or builds upon one or more prior RCTs.
Abstract
This study examines the effectiveness of communication and monitoring strategies in leveraging past evaluation results to drive school improvement efforts in the Dominican Republic. The trial is embedded within the country’s preparation for the 2025 high school diagnostic evaluation and targets both district and school-level interventions. The primary objectives include enhancing the use of 2019 diagnostic evaluation data, improving administrative compliance, and fostering the implementation of evidence-based improvement plans.

The intervention includes randomized assignments to districts and schools with varying intensities of communication and monitoring, enabling the evaluation of their impact on intermediate outcomes such as contact information updates, report usage, and improvement plan submissions, as well as on educational outcomes. District supervisors receive detailed reports on the compliance levels of the schools in their jurisdiction, enabling targeted follow-ups and monitoring. School directors receive tailored email campaigns prompting actions such as reviewing past results and submitting improvement plans. Districts are randomized into three groups based on the frequency of reporting: minimal (3 reports), moderate (4 reports), and intensive (6 reports). Schools within these districts are further randomized into two communication levels: minimal (3 emails) and intensive (6 emails ).

Outcomes are evaluated across intermediate benchmarks such as compliance with form submissions and contact information updates, and final educational performance indicators, including average student diagnostic scores and the proportion of students achieving different knowledge levels (elemental, acceptable, satisfactory) in the 2025 evaluation. The findings will provide insights into the role of strategic communication and monitoring in fostering educational accountability and improvement.
External Link(s)

Registration Citation

Citation
Morales, Daniel and Catherine Rodríguez. 2024. "Maximizing the Impact of Analyzing Previous Educational Results: Communication and Monitoring for School Improvement." AEA RCT Registry. December 20. https://doi.org/10.1257/rct.14955-1.0
Experimental Details

Interventions

Intervention(s)
The intervention involves sending reports to district supervisors and tailored emails to school directors to prompt the analysis of the 2019 diagnostic evaluation results and planning for the 2025 evaluation as ordered by the Ministry of Education. Districts are randomized into three groups with varying frequencies of reporting: minimal (3), moderate (4), and intensive weekly reports (6). Schools within districts are further randomized into two reminder communication levels: minimal (3) and intensive (6). Outcomes include compliance with updating contact details, submitting teacher improvement plans, and overall improvement in diagnostic results.
Intervention Start Date
2024-12-11
Intervention End Date
2025-05-31

Primary Outcomes

Primary Outcomes (end points)
1. The percentage of schools updating contact information.
2. Proportion of school directors completing required forms.
3. Percentage of teachers submitting improvement plans.
4. Average diagnostic scores in 2025 national standardized diagnostic test.
5. Proportion of students achieving different performance levels in the 2025 national standardized diagnostic test.
Primary Outcomes (explanation)
For intermediate outcomes:
• Updating Contact Information: Measured as the percentage of missing contacts filled.
• Completion of Forms: Binary variable indicating whether a school director submitted the form of assigned teachers to design and implement improvement plans.
• Submission of Improvement Plans: Percentage of teachers uploading improvement plans by the deadline.

For long-term outcomes:
• Average Diagnostic Scores: Derived from student-level data in 2025 evaluation reports.
• Proportion Satisfactory: Share of students achieving different performance levels (elemental, acceptable, satisfactory) in key subjects.

Secondary Outcomes

Secondary Outcomes (end points)
• Email engagement rates (open and click-through rates).
• Comparison of compliance between different treatment arms at district and school levels.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The experiment employs a two-tier randomization design:
1. District Level: Districts are randomized into three groups with varying levels of reporting frequency:
o Group 1: Minimal (3 emails reports).
o Group 2: Moderate (4 emails reports).
o Group 3: Intensive (6 emails reports).

2. School Level: Schools within each district are further randomized into:
o Group A: Minimal communication (3 emails).
o Group B: Intensive communication (6 emails).
Experimental Design Details
Not available
Randomization Method
Randomization will be conducted by a computer using a stratified approach to balance districts based on the proportion of valid emails, 2019 average scores, and satisfactory performance levels.
Randomization Unit
• Clusters: Districts for the first level.
• Sub-clusters: Schools within districts for the second level.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
Planned Number of Clusters: 122 districts.
Planned Number of Observations: 2,468 schools.
Sample size: planned number of observations
Districts: • Group 1: Minimal Reports — 41 districts. • Group 2: Moderate Reports — 41 districts. • Group 3: Intensive Reports — 40 districts. Schools: • Group A: Minimal Communication — 1234 schools. • Group B: Intensive Communication — 1234 schools. Students: more than 135K.
Sample size (or number of clusters) by treatment arms
• Group 1 (41 districts): Minimal Reports
o Schools Group A: Minimal Communication — 414 schools.
o Schools Group B: Intensive Communication — 412 schools.
• Group 2 (41 districts): Moderate Reports
o Schools Group A: Minimal Communication — 425 schools.
o Schools Group B: Intensive Communication — 444 schools.
• Group 3 (40 districts): Intensive Reports
o Schools Group A: Minimal Communication — 395 schools.
o Schools Group B: Intensive Communication — 378 schools.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Dirección de Evaluación de los Aprendizajes - MINERD
IRB Approval Date
2024-11-21
IRB Approval Number
N/A