Nudging Freshmen to Persist: Evidence from Two Large Private Colleges in Brazil

The intervention is a student success initiative developed by Movva, an ed-tech company that applies behavioral science and artificial intelligence to improve retention in higher education. Movva identifies students at risk of dropping out and sends them personalized, behaviorally-informed messages through digital channels such as SMS, WhatsApp, and push notifications. These communications are designed to support motivation, promote academic engagement, and encourage course completion.

The initiative was implemented in partnership with Vitru Educação, a major private higher education provider in Brazil, beginning in the first semester of 2024. It targeted approximately 60,000 first-semester undergraduate students per semester, across a broad set of programs and campuses ("polos") nationwide. The intervention was designed to be both personalized and scalable, adapting messages to student profiles while reaching a large and geographically diverse population.

The intervention is a student success initiative developed by Movva, an ed-tech company that applies behavioral science and artificial intelligence to improve retention in higher education. Movva identifies students at risk of dropping out and sends them personalized, behaviorally-informed messages through digital channels such as SMS, WhatsApp, and push notifications. These communications are designed to support motivation, promote academic engagement, and encourage course completion.

The initiative was implemented in partnership with Vitru Educação, a major private higher education provider in Brazil, beginning in the first semester of 2024. It targeted approximately 60,000 first-semester undergraduate students per semester, across a broad set of programs and campuses ("polos") nationwide. The intervention was designed to be both personalized and scalable, adapting messages to student profiles while reaching a large and geographically diverse population.

2024-01-31

2024-12-31

1. Student Retention and Persistence

1. Student Retention and Persistence
Dropout or withdrawal status during, shortly after the intervention or/and by the end of the academic semester.

1. Academic Performance
2. Tuition Payments and Revenue
3. Heterogeneous effects by Age
4. Heterogeneous effects by treatment dose (variation in the intensity of the treatment)
5. Heterogeneous effects by region

1. Academic Performance
This outcome will be constructed using students' grades in each enrolled course and their final course grades during the semester. The measure will reflect both continuous performance and course completion.
Additional measures may include:
Number of passed courses (binary per course, then summed)
GPA average (normalized if grading scales vary across programs)

2. Tuition Payments and Revenue
Monthly tuition payments made by each student.
Total tuition revenue received per student during the semester.
Payment regularity or delinquency

3. Heterogeneous Effects by Age
We will estimate treatment effect heterogeneity based on students’ age at the start of the semester. To investigate whether the intervention is more effective for younger students (likely recent high school graduates) or older ones (potentially returning or working students).

4. Heterogeneous Effects by Treatment Dose
This captures variation in effects due to the intensity or type of messaging students received.
Use experimental group assignment:
SMS-only vs. SMS + WhatsApp vs. Push-only
Create dose indicators based on number of messages successfully delivered (if available)
Explore whether increased exposure (e.g., 6 vs. 8 messages/month) correlates with outcomes

5. Exploratory Analysis: Regional Variation in Treatment Effects
While not a pre-specified subgroup analysis, we will explore whether treatment effects vary meaningfully by region or institutional campus (“polo”), given possible differences in local infrastructure, student composition, or program delivery. Regional differences were considered in the randomization design to minimize contamination, exploration of treatment heterogeneity by region may reveal contextual moderators of effectiveness.

In the first semester of 2024, a randomized intervention was implemented to evaluate the effectiveness of different communication channels in promoting student success among freshmen college students. The selection strategy prioritized courses with the largest student enrollments, and within each course, students were assigned to treatment or control groups based on their campus location. The intervention targeted approximately 60,000 students, divided across three experimental arms. One control group received no messages, while two treatment groups were exposed to different communication strategies: one group received eight SMS messages per month (two per week), and the other received a combination of six SMS and two WhatsApp messages per month.

In the second semester of 2024, the intervention design was refined based on findings from the earlier semester, with a stronger focus on cost-effective and scalable communication strategies. Given budget constraints, the most successful strategy from 2024-1—combining SMS and WhatsApp—was prioritized for a treatment group of 13,443 students. These students were drawn from eight courses that had shown the most promising results previously. A corresponding control group of 9,701 students from the same courses did not receive any messages, allowing for a clean comparison. In addition, two new experimental groups were introduced to test the effectiveness of push notifications, a low-cost communication channel. The first push group included 4,590 students and was designed to assess the impact of push messages alone. A second, broader push notification group included 17,823 additional students who had not been part of the original sample, allowing researchers to explore the channel’s scalability and performance across a more diverse student population. This multi-arm design aimed to balance rigor with practical considerations and expand the evaluation of digital messaging in higher education.

In the second semester of 2024, two distinct randomized experiments were conducted to evaluate the effectiveness of digital communication strategies in promoting the success of freshmen college students. Experiment 1 focused on two treatment arms and a control group, drawing on evidence from the most promising courses in the previous semester. A total of 27,734 students were included in this experiment. The first treatment group, comprising 13,443 students, received eight messages per month—six via SMS and two via WhatsApp. The second treatment group included 4,590 students who received messages exclusively through push notifications, a no-cost communication channel. A control group of 9,701 students received no messages. This design allowed for a direct comparison between two delivery strategies (SMS+WhatsApp vs. push notifications) and a control condition, all within a high-priority student population.

Experiment 2 aimed to expand the evaluation of push notifications to a broader and more diverse population of freshmen college students, leveraging the low cost of this communication channel. This experiment included a total of 36,916 students, randomly assigned to one of two groups. The treatment group consisted of 17,823 students who received messages exclusively via push notifications. The control group, comprising 19,093 students, received no messages. Unlike Experiment 1, which focused on a smaller set of high-priority courses, Experiment 2 included a wider range of academic programs and campuses. This design enabled the research team to assess the scalability and generalizability of push notifications as a tool for improving student outcomes in varied institutional contexts.

In the first semester of 2024, a randomized intervention was implemented to evaluate the effectiveness of different communication channels in promoting student success among freshmen college students. The selection strategy prioritized courses with the largest student enrollments. Within each selected course, students were assigned to treatment or control groups based on their campus location (“polo”), a strategy intended to reduce contamination across experimental arms by ensuring geographic separation. To construct the sample, the research team used the enrollment base from the second semester of 2023 as a reference, extrapolating it by a factor of three to estimate the expected 2024-1 intake of 250,000 to 300,000 students while preserving the original distribution across courses and campuses. Only campuses with at least 24 students were eligible, and each treatment or control group within a course needed to include at least 500 students. The final sample included approximately 60,000 students, distributed across three experimental arms. One control group received no messages. One treatment group received eight SMS messages per month (two per week), while another treatment group received a hybrid intervention composed of six SMS messages and two WhatsApp messages per month. An initially planned push notification arm was excluded due to technical issues. The communication period extended from January to July 2024.

In the second semester of 2024, the intervention design was refined based on operational insights and preliminary evidence from 2024-1, with a stronger focus on cost-effective and scalable communication strategies. Two distinct randomized experiments were carried out. To construct the sample for Experiment 1, the research team selected students from eight academic programs that had demonstrated the strongest positive impacts in the first-semester experiment. A total of 27,734 students were included in this experiment. Students were randomly assigned to one of three groups: 13,443 students in the first treatment group received the best-performing strategy from 2024-1—eight messages per month combining six SMS and two WhatsApp; 4,590 students in the second treatment group received only push notifications; and 9,701 students served as the control group, receiving no messages. This structure enabled a focused comparison of delivery channels while maintaining high internal validity within a prioritized, evidence-based subset of the student body.

Experiment 2 was designed to extend the reach of the intervention to a broader and more heterogeneous set of incoming students, making use of the low cost of push notifications. The sampling frame included freshmen who were not part of Experiment 1, allowing the study to capture variation across a wider range of courses and campuses. A total of 36,916 students were randomly assigned to one of two groups: 17,823 students in the treatment group received messages exclusively via push notifications, while 19,093 students in the control group received no messages. This design allowed for the assessment of the generalizability and scalability of push notifications as a low-cost digital strategy for student engagement.

In the first semester of 2024, a randomized intervention was implemented to evaluate the effectiveness of different communication channels in promoting student success among freshmen college students. The selection strategy prioritized courses with the largest student enrollments. Within each selected course, students were assigned to treatment or control groups based on their campus location (“polo”), a strategy intended to reduce contamination across experimental arms by ensuring geographic separation. To construct the sample, the research team used the enrollment base from the second semester of 2023 as a reference, extrapolating it by a factor of three to estimate the expected 2024-1 intake of 250,000 to 300,000 students while preserving the original distribution across courses and campuses. Only campuses with at least 24 students were eligible, and each treatment or control group within a course needed to include at least 500 students. The final sample included approximately 60,000 students, distributed across three experimental arms. One control group received no messages. One treatment group received eight SMS messages per month (two per week), while another treatment group received a hybrid intervention composed of six SMS messages and two WhatsApp messages per month. An initially planned push notification arm was excluded due to technical issues. The communication period extended from January to July 2024.

In the second semester of 2024, the intervention design was refined based on operational insights and preliminary evidence from 2024-1, with a stronger focus on cost-effective and scalable communication strategies. Two distinct randomized experiments were carried out. To construct the sample for Experiment 1, the research team selected students from eight academic programs that had demonstrated the strongest positive impacts in the first-semester experiment. A total of 27,734 students were included in this experiment. Students were randomly assigned to one of three groups: 13,443 students in the first treatment group received the best-performing strategy from 2024-1—eight messages per month combining six SMS and two WhatsApp; 4,590 students in the second treatment group received only push notifications; and 9,701 students served as the control group, receiving no messages. This structure enabled a focused comparison of delivery channels while maintaining high internal validity within a prioritized, evidence-based subset of the student body.

Experiment 2 was designed to extend the reach of the intervention to a broader and more heterogeneous set of incoming students, making use of the low cost of push notifications. The sampling frame included freshmen who were not part of Experiment 1, allowing the study to capture variation across a wider range of courses and campuses. A total of 36,916 students were randomly assigned to one of two groups: 17,823 students in the treatment group received messages exclusively via push notifications, while 19,093 students in the control group received no messages. This design allowed for the assessment of the generalizability and scalability of push notifications as a low-cost digital strategy for student engagement.

Randomization was conducted at the class or section code associated with the student's enrollment.

Yes

First semester of 2024: 18183 sections
Second semester of 2024: 26564 sections

First semester of 2024: 67,654 students Second semester of 2024: 62,818 students

First semester of 2024: 67,654 students (34206 control - 33448 treatment)
Second semester of 2024: 62,818 students (27,968 control - 34,850 treatment)

Given the large sample size (over 60,000 units per semester) and substantial number of clusters (approximately 20,000 per semester), the study is well-powered to detect even small effect sizes in the main outcomes. Accounting for the clustered sample design, the minimum detectable effect size is expected to be small relative to the observed standard deviations of key outcomes. This level of statistical power allows us to identify even modest impacts with a high degree of confidence, ensuring robust and policy-relevant findings.