Abstract
This RCT provided encouragement and information nudges to take subsequent economics courses and major in the subject for students enrolled in large introductory economics classes (microeconomics, macroeconomics, and statistics) at the University of Illinois at Urbana-Champaign, a large "elite public" university. The aim of the RCT was to investigate gender differences in response to information, encouragement to major, and alternative framings of the usefulness of economics.
Students enrolled in introductory classes in Fall 2015, Spring 2016, and Fall 2016 were offered a small amount of extra credit for taking a survey about their backgrounds and interests in various majors. Students were immediately solicited for enrollment in the study upon completion of the survey. Consent and participation were nearly universal. There were 1,976 consenting students who were Freshmen or Sophomores and met the minimum grade requirements (at least a B- grade for the "focal course", or course of study enrollment, and an overall GPA of at least 2.67) to be randomized to control and treatments. 1,134 were men and 842 were women. These students were randomized in equal proportions (1/3 each) to control and the two treatment arms.
Treated students received a communication from the department via postal mail and email shortly after the end of the semester, once the course had closed and they had seen their course grade. The first and third paragraphs of the cover letter were the same for all students receiving any treatment. The first paragraph offered encouragement and reassurance on ability (“you have the potential for success in Economics…consider adding an economics major”). The final paragraph presented general information about the department (offering “many activities and experiences to build on your academic training…mentoring and tutoring supports”) and provided contact information for assistance with major requirements and course scheduling. The second paragraph differed across the treatments. In the Prosocial Treatment, a persuasive statement framed economics as a broad way of thinking that is applicable to many issues outside of the business realm, such as “helping to combat climate change, teaching children in the classroom, motivating people to adopt healthy behaviors, working to promote self-sufficiency in developing countries, even raising your own kids.” The framing and persuasion paragraph for the Earnings Treatment conveyed that “Economics is one of the best-rewarded majors in the job market” with “pre-graduation job offers at the second-highest rate of all majors…[and] among the highest mid-career earnings.”
The two treatment communications had differing attachments with supporting evidence on the framings. The attachment to the Prosocial Treatment letter provided information on other-regarding career opportunities for economics majors, including specific examples of jobs held by famous economics majors and University of Illinois alumni economics majors, plus evidence of job opportunities relevant to the framing. An attachment to the Earnings Treatment letter provided examples of jobs in the business sector taken by the Economics Department’s graduates and information on pay trajectories and frequency of pre-graduation job offers for students with an economics major relative to other majors in general.
Students were matched to University administrative records in order to observe grades earned in the focal introductory course, any economics courses taken in semesters subsequent to to study enrollment, and major and minor declarations. Students were observed in the administrative data through the end of their Junior year.
The analysis focused on differential responses to treatment in course-taking and economics major declarations by gender. Whether students responded to treatment and whether students responded differently to the alternative framings across the two treatment arms was examined. Two potential mediators of treatment suggested by the prior literature--the difference between expected and realized focal course grade and presence of a female teaching assistant (TA)--were also examined.
Descriptive analysis revealed that men in the study were more likely to be U.S. citizens, had higher pre-college math ability, were marginally more likely to have already declared an economics major at solicitation, and expressed a greater interest in the economics major at enrollment. There was not a significant difference between men and women in expected grade in the focal course. Two-thirds (66.3%) of the Freshmen and Sophomore students took another course after the focal course and 22.2% of students who declared a major by the end of their Junior year chose the economics major.
There was strong evidence that samples were balanced across the control and treatments (equality was rejected in only 2 out of 34 tests of equality of means between the control sample and each treatment sample). Ordinary Least Squares was used to estimate treatment effects, and standard errors were robustly estimated. An extensive set of background characteristics (including intensity of interest in the economics major at study enrollment, academic performance, and course and semester-year of study enrollment) were included in all regression specifications. The first specification, estimated for the entire sample, included treatment indicators, gender indicators, and their interaction to estimate treatment effects on men and women. Two mediated treatment specifications were estimated separately for men and women. In addition to the mediator and treatment variables, these specifications included an interaction of the mediator with each treatment.
Estimation of basic (i.e., unmediated) treatment effects revealed weak evidence of a positive effect of the Prosocial Treatment on men taking another economics course (0.053 se=0.030). The effect of the Earnings Treatment (0.032 se=0.013) was positive at standard confidence levels. However, the hypothesis that the two treatment effects differed from each other was rejected at standard confidence levels. The effects on majoring in economics were positively-signed for the Prosocial Treatment (0.041 se=0.032), and the point estimate was closer to zero for the Earnings Treatment (0.012 se=0.029). All coefficients on the interactions of female and treatment were negative in sign, and there was no evidence of a positive nudge to either take a subsequent course or major in economics for women. The point estimate of the total effect of the earnings treatment on declaring an economics major for women (0.041-0.047=-0.007) was close to zero.
41.5% of women and 42.6% of men received a lower-than-expected grade, while 11.2% of women and 7.2% of men received a higher-than-expected grade. The findings for the grade-mediated treatments were as follows. Either treatment nudged women who received a lower-than-expected grade away from taking another course (-0.162 se=0.055 for Prosocial Treatment and -0.152 se=0.052 for Earnings Treatment). The total reduction in the probability of taking an additional course due to treatment for women with lower-than-expected grades was -0.102 (p=0.030) and -0.101 (p=0.065) for Prosocial and Earnings Treatments, respectively. In contrast, there was a positive total effect of the Earnings Treatment on men with lower-than-expected grades (0.060 p=0.057). Although insignificant at standard confidence levels, the net effects of both treatments for students with higher-than-expected grades were positive in sign and often roughly similar in magnitude for both men and women.
The differential effects of having a female TA on men and women were examined. For both treatments, and for both men and women, the absolute values of the interaction coefficients were large in absolute magnitude (10 percentage points or greater). However, they were opposite-signed for men (positive) and women (negative). The total effects of the two treatment arms on men with a female TA were positive at standard confidence levels for the Prosocial and Earnings Treatments, but total effects of treatment on women were not estimated to differ from zero at reasonable confidence levels.
The RCT findings reached three major conclusions. First, considering all of the evidence, men were overall more readily nudged to pursue further study of economics than women, regardless of the type of nudge (Earnings Treatment or Prosocial Treatment). This reinforced some findings in the prior literature that men were more favorably inclined in general to pursue economics. Second, the hypothesis that the two treatments had differential effects (for either women or men) was always rejected at standard confidence levels. This result differed from prior literature finding that a "business" framing of economics reduced women's majoring. Third, findings on mediators of treatment differed markedly for men and women. Women receiving grades in the focal course that were below their expectations responded negatively to encouragement to pursue economics, regardless of framing (in contrast, there was suggestive evidence that men with below-expected grades might be nudged into economics by the Earnings Treatment). This finding is in the spirit of several studies that found women's grade sensitivity was an important factor in their failure to progress beyond introductory courses. The hypothesis that female TAs heightened treatment effects on women was rejected at standard confidence levels. In contrast, there was some evidence that male college students were more responsive to treatment when their TA was a woman. While prior research found women role models encouraged women to major in economics, the role models used in these studies were not TAs.