From natural variation to optimal policy? The importance of endogenous peer group formation.

Last registered on July 27, 2016
View Trial History

Pre-Trial

Trial Information

General Information

Title
From natural variation to optimal policy? The importance of endogenous peer group formation.
RCT ID
AEARCTR-0001446
Initial registration date
July 27, 2016

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
July 27, 2016, 8:04 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
United States of America
Region
North America

Primary Investigator

Name
Bruce Sacerdote
Affiliation
Dartmouth College
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
James E. West
PI Affiliation
Dept. of Economics, Baylor University
Contact Investigator
PI Name
Scott E. Carrell
PI Affiliation
Professor of Economic, UC Davis
Contact Investigator

Additional Trial Information

Status
Completed
Start date
2007-08-01
End date
2012-12-31
Keywords
Education
Additional Keywords
Peer effects, social network formation, homophily, United States
JEL code(s)
Secondary IDs
Abstract
We take cohorts of entering freshmen at the United States Air Force Academy and assign half to peer groups designed to maximize the academic performance of the lowest ability students. Our assignment algorithm uses nonlinear peer effects estimates from the historical pre-treatment data, in which students were randomly assigned to peer groups. We find a negative and significant treatment effect for the students we intended to help. We provide evidence that within our “optimally” designed peer groups, students avoided the peers with whom we intended them to interact and instead formed more homogeneous subgroups. These results illustrate how policies that manipulate peer groups for a desired social outcome can be confounded by changes in the endogenous patterns of social interactions within the group.
External Link(s)
J-PAL Evaluation Summary

Registration Citation

Citation
Carrell, Scott E., Bruce Sacerdote and James E. West. 2016. "From natural variation to optimal policy? The importance of endogenous peer group formation.." AEA RCT Registry. July 27. https://doi.org/10.1257/rct.1446-1.0
Former Citation
Carrell, Scott E., Bruce Sacerdote and James E. West. 2016. "From natural variation to optimal policy? The importance of endogenous peer group formation.." AEA RCT Registry. July 27. https://www.socialscienceregistry.org/trials/1446/history/9757
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The United States Air Force Academy organizes its freshman class into squadrons of approximately 30 students each according to an algorithm that creates demographically balanced peer-groups. Researchers conducted a randomized evaluation to test the impact of an alternate algorithm that optimizes the performance of lower-ability students instead.

Incoming students in 2007 and 2008 were randomly assigned to either a control or treatment group. The control group squadrons were assigned using the existing algorithm. The treatment group squadrons were assigned as specially engineered peer groups. Researchers analyzed the squadron composition and academic performance of first-year students from 2001-2006 and, based on these data, generated a formula that sorted students into groups predicted to maximize the performance of the lower-ability students. The application of this formula resulted in the creation of two types of squadrons: one type composed largely of lower-ability and high-ability students only (bimodal squadron), and the other composed largely of middle-ability students (homogenous squadrons). Students remained in their squadrons for the first year.
Intervention Start Date
2007-08-01
Intervention End Date
2012-06-30

Primary Outcomes

Primary Outcomes (end points)
GPA at the end of freshman year
Primary Outcomes (explanation)
GPA information was collected from the Academy

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
A sorting algorithm was created using pre-treatment data that sorted incoming freshmen into squadrons meant to optimize the grade point of the bottom third of low-ability students. Freshmen all take the same required courses during their first-year, take common exams to measure achievement, and have no choice as to their professors thus ensuring there is no self-selection of students into courses or certain professors. Incoming students in the fall of 2007 and 2008 were randomly allocated to control or treatment arms. Control students were assigned to squadron based on the existing algorithm that oprtimized demographic diversity. Treatment students were sorted using the new algorithm.

Researchers used administrative data from the Academy track academic performance. Researchers also surveyed students in the spring of their sophomore and junior years about their study partners and friends.
Experimental Design Details
Randomization Method
The random selection of the treatment and control squadrons was stratified across the four cadet “groups” which contain 10 squadrons each. It was also stratified with respect to new and returning “Air Officers Commanding” or AOCs, the officer in charge of military training within each squadron. This was done to eliminate any potential group or AOC level common shocks to academic performance. We flipped the treatment and control squadrons after the first year of the experiment. Additionally, the random division was subject to the constraint that siblings were split between the treatment and control groups.
Randomization Unit
Individual students
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
Study not clustered
Sample size: planned number of observations
2653 students
Sample size (or number of clusters) by treatment arms
about 80 squadrons from two cohorts at the US Air Force Academy
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
US Air Force Academy
IRB Approval Date
2007-03-01
IRB Approval Number
FAC2007030H
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
June 30, 2012, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
December 31, 2012, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
Study not clustered
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
2447 students
Final Sample Size (or Number of Clusters) by Treatment Arms
Treatment: 1228 students Control: 1219 students
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
No
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
FROM NATURAL VARIATION TO OPTIMAL POLICY? THE IMPORTANCE OF ENDOGENOUS PEER GROUP FORMATION

Abstract: We take cohorts of entering freshmen at the United States Air Force Academy and assign half to peer groups designed to maximize the academic performance of the lowest ability students. Our assignment algorithm uses nonlinear peer effects estimates from the historical pre-treatment data, in which students were randomly assigned to peer groups. We find a negative and significant treatment effect for the students we intended to help. We provide evidence that within our “optimally” designed peer groups, students avoided the peers with whom we intended them to interact and instead formed more homogeneous subgroups. These results illustrate how policies that manipulate peer groups for a desired social outcome can be confounded by changes in the endogenous patterns of social interactions within the group.
Citation
Carrell, Scott E., Bruce I. Sacerdote, and James E. West. 2013. "From Natural Variation to Optimal Policy? The Importance of Endogenous Peer Group Formation." Econometrica 81(3), 855-882.
URL
https://www.povertyactionlab.org/sites/default/files/publications/990%20Endogenous%20Peer%20Group%20Formation%202013_0.pdf

Reports & Other Materials