Experimental Design
Research Design:
To investigate the causal relationship between the four candidate frictions and 401(k) plan engagement, we administered a field experiment, embedded in an online survey, to undersaving employees at a large US financial services firm. We marketed the online instrument as an employer-sponsored opportunity for employees to provide confidential feedback regarding the workplace and benefit programs. The first module of the instrument featured a survey that, beyond capturing relevant background information, assessed the employee-specific incidence of each candidate friction. The second module promised employees to assess their retirement preparedness and, if necessary, provide guidance to improve their preparedness. This module implemented the field experiment by randomizing employees to one of several information- and/or incentive-based treatments. While the survey provided evidence as to the baseline prevalence of each friction and the field experiment provided evidence as to the average effect of reducing/engaging each friction on saving, jointly, the survey and field experiment clarify the heterogeneous importance of frictions across baseline incidence.
Sample Construction:
On July 19, 2016, we invited a pre-specified sample of undersaving employees of low-to-moderate income by email to participate in a ten- to fifteen-minute survey marketed as an opportunity to provide confidential feedback on the workplace and employee benefit programs. The email explained that while the survey was part of a broader partnership with the firm to help improve employee well-being, it was independently designed and administered by academic researchers from Carnegie Mellon University. Employees were directed to participate in the survey, hosted on the Qualtrics platform, by clicking a personalized link within the ten-day survey period. To encourage a high response rate, survey respondents were entered into a raffle for an Apple iPad and were reminded to complete the survey via email.
The construction of the invitation sample was shaped by two considerations—a desire to target undersaving employees of low-to-moderate income and the firm’s request to limit invitations to 5,000 employees. To differentiate between employees who did and did not fully claim the plan match, we ultimately invited two non-overlapping samples of undersaving employees to the survey. The primary sample (the “Low Arm”) comprised the universe of 3,719 401(k) plan-eligible employees who, as of late June 2016, were 25 to 55 years of age, earned less than $100k annually and contributed less than 4 percent to their 401(k) plan (inclusive of non-participants). A second sample of 1,000 (the “Moderate Arm”) comprised a random draw of plan-eligible employees who, as of late June 2016, were 25 to 55 years of age, earned less than $100k annually and contributed 4 to 9 percent to their 401(k) plan. Twenty-eight percent of invited employees participated in the study. We attribute the relatively high response rate for an email solicitation to the lottery-based incentive, email reminder, and sponsorship/promotion by the employer. After excluding those who exited the survey prematurely or were already contributing at or above their recommended rate, we randomly assigned 1,137 employees to an experimental treatment within one of two study arms (780 in the Low Arm; 357 in the Moderate Arm).