Beacon Mentoring
Last registered on March 14, 2017

Pre-Trial

Trial Information
General Information
Title
Beacon Mentoring
RCT ID
AEARCTR-0002066
Initial registration date
March 14, 2017
Last updated
March 14, 2017 7:06 PM EDT
Location(s)
Primary Investigator
Affiliation
MDRC
Other Primary Investigator(s)
Additional Trial Information
Status
Completed
Start date
2008-01-15
End date
2010-02-01
Secondary IDs
Abstract
The Beacon Mentoring program was designed and implemented by South Texas College, in McAllen, Texas. As part of their participation in the Achieving the Dream initiative, college leaders developed an innovative intervention targeting students enrolled in lower-level math courses that have high rates of failure. The program was based on three simple ideas: that students who need services often do not access them even when they are available and free; that a mechanism is needed to alert student services staff when students start to fail in class; and that students need a “go to” person on campus who knows them and to whom they can turn for advice, support, and information.

The Beacon Mentoring program was a “light touch” intervention, designed to serve large numbers of students at minimal cost. Mentors were college employees who were recruited and trained to make several short classroom presentations about services available on campus and to work with the faculty to identify struggling students and offer them help early on. During spring 2008, the Beacon program targeted over 2,000 students enrolled in 83 sections of either a developmental (remedial) math course or a college-level algebra course. Mentors were randomly assigned to half of the sections. The impact of the program was assessed by comparing the outcomes of students in the mentored classes with the outcomes of students in the classes that were not assigned a mentor.

The program had no effect on passing the math class or on persistence. However, evidence suggests that the program had a modest and positive impact on other outcomes for the full sample of students and resulted in additional or more pronounced benefits for two subgroups of students most at risk of failure: students enrolled in developmental math and students who attended college part time. The findings show that:
-- The program succeeded in increasing the number of students who used the Center for Learning Excellence, a campus resource that provided tutoring and other forms of academic support.
-- The program led to a statistically significant decrease in the likelihood of students in mentored classes withdrawing from their math course before the end of the semester.
-- Part-time students were less likely to withdraw from and more likely to pass the math class, earned more credits, and, at least in the developmental math classes, scored higher on the final exam. The program also resulted in enhanced benefits for students enrolled in developmental math classes.
External Link(s)
Registration Citation
Citation
Visher, Mary. 2017. "Beacon Mentoring." AEA RCT Registry. March 14. https://www.socialscienceregistry.org/trials/2066/history/15003
Experimental Details
Interventions
Intervention(s)
The Beacon Mentoring Program consists of three elements:

1. Delivering information directly to students in classrooms about academic support services, advising and counseling services, financial aid, and Priority Registration (early registration for the following semester available to a targeted group of students, including those in the Beacon program)
2. Providing each student with a “go to” person on campus who knows them and is trained to offer certain kinds of assistance
3. Improving communication between faculty and student services professionals so that they can take immediate steps to help students before they fail or drop out of class
Intervention Start Date
2008-01-21
Intervention End Date
2008-05-09
Primary Outcomes
Primary Outcomes (end points)
Math course pass rate, overall pass rate, persistence in college
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
Between October 2007 and early January 2008, over 2,100 students enrolled as they normally would in 83 sections of the three targeted math courses for the spring semester. The 6math courses were Math 80 and 85, the two lowest levels of developmental math, and Math 1414, a college-level algebra course. Between 20 and 35 students were enrolled in each section of these three courses. All faculty members who were assigned to teach these sections had agreed to participate in the study and understood that each section they taught had about a 50 percent chance of being assigned a Beacon mentor. In mid-January, during the first week of class, MDRC randomly assigned these 83 sections to either the program group (where the section was assigned a mentor) or the control group (where no mentor was assigned). The procedure used to perform the random assignment ensured that multiple sections taught by the same instructor were evenly divided between the program and control groups, thus effectively controlling for teacher effects. Day and evening classes were also evenly divided between the program and control groups. On the same day that MDRC informed the college of the results of random assignment, college leaders assigned mentors to each section. Decisions about which mentor to assign to which class were mostly mechanistic, but in some cases mentors were assigned to a class taught by an instructor with whom they had collaborated earlier during a pilot semester for the program.
Experimental Design Details
Randomization Method
The Beacon Mentoring program was implemented for three levels of math: Math 0080 (Basic Mathematics), Math 0085 (Introductory Algebra) and Math 1414 (College Algebra). Math 0080 and Math 0085 are developmental courses, and Math 1414 is the first level of college math. Math class sections were randomly assigned to the program or control group as follows.

The developmental courses (Math 0080 and 0085) were put in a separate block from the college-level course (Math 1414). Those blocks were then split up by teacher. Finally, these blocks were split up by time of class — daytime or evening. This last partition yielded the final RA blocks that were used for randomly assigning treatment sections of the math classes. Once the RA blocks were defined, the first section in each block was randomly given a “P” or “C” for program or control. The subsequent sections within each block then alternated status. So, for example, teachers who had only one section would essentially get a coin flip for their assignment. If they had two sections during the daytime, they were guaranteed to get one of each assignment status.
Randomization Unit
Class section
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
83 class sections
Sample size: planned number of observations
2,165 students
Sample size (or number of clusters) by treatment arms
42 class sections control, 41 class sections mentored
(1,098 students control, 1,067 students in mentored classes)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
IRB Approval Date
IRB Approval Number
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
Yes
Intervention Completion Date
May 09, 2008, 12:00 AM +00:00
Is data collection complete?
Yes
Data Collection Completion Date
Final Sample Size: Number of Clusters (Unit of Randomization)
83 class sections
Was attrition correlated with treatment status?
Final Sample Size: Total Number of Observations
2,165 students
Final Sample Size (or Number of Clusters) by Treatment Arms
42 class sections control, 41 class sections mentored
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
No
Reports and Papers
Preliminary Reports
Relevant Papers
Abstract
This report presents results from a rigorous evaluation of the Beacon Mentoring program, designed and implemented by South Texas College, in McAllen, Texas. As part of their participation in the Achieving the Dream initiative, college leaders developed an innovative intervention targeting students enrolled in lower-level math courses that have high rates of failure. The program was based on three simple ideas: that students who need services often do not access them even when they are available and free; that a mechanism is needed to alert student services staff when students start to fail in class; and that students need a “go to” person on campus who knows them and to whom they can turn for advice, support, and information.

The Beacon Mentoring program was a “light touch” intervention, designed to serve large numbers of students at minimal cost. Mentors were college employees who were recruited and trained to make several short classroom presentations about services available on campus and to work with the faculty to identify struggling students and offer them help early on. During spring 2008, the Beacon program targeted over 2,000 students enrolled in 83 sections of either a developmental (remedial) math course or a college-level algebra course. Mentors were randomly assigned to half of the sections. The impact of the program was assessed by comparing the outcomes of students in the mentored classes with the outcomes of students in the classes that were not assigned a mentor.

The program had no effect on passing the math class or on persistence. However, evidence suggests that the program had a modest and positive impact on other outcomes for the full sample of students and resulted in additional or more pronounced benefits for two subgroups of students most at risk of failure: students enrolled in developmental math and students who attended college part time. Analyses in this report show that:
-- The program succeeded in increasing the number of students who used the Center for Learning Excellence, a campus resource that provided tutoring and other forms of academic support.
-- The program led to a statistically significant decrease in the likelihood of students in mentored classes withdrawing from their math course before the end of the semester.
-- Part-time students were less likely to withdraw from and more likely to pass the math class, earned more credits, and, at least in the developmental math classes, scored higher on the final exam. The program also resulted in enhanced benefits for students enrolled in developmental math classes.
Citation
Visher, Mary, Kristin F. Butcher, and Oscar Cerna. 2010. Guiding Developmental Math Students to Campus Services: An Impact Evaluation of the Beacon Program at South Texas College. New York: MDRC.