’I Won’t Get it Anyway’ - A Field Study on Misperceptions About Financial Student Aid and Reasons for Non-take-up

Last registered on August 21, 2024

Pre-Trial

Trial Information

General Information

Title
’I Won’t Get it Anyway’ - A Field Study on Misperceptions About Financial Student Aid and Reasons for Non-take-up
RCT ID
AEARCTR-0011249
Initial registration date
April 24, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 03, 2023, 4:03 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
August 21, 2024, 5:13 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Max Planck Institute for Research on Collective Goods

Other Primary Investigator(s)

Additional Trial Information

Status
Completed
Start date
2023-05-02
End date
2023-12-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
I conduct an online survey experiment to investigate the relationship between misperceptions about social benefit program criteria, their correction, and take-up. The study focusses on the German federal financial student aid (BAföG) and elicits students’ misperceptions about the program criteria through hypothetical scenarios. Half of the participants receive bundled information designed to resolve possible misperceptions at the end of the survey. In a follow-up survey six months later, students are asked about their application status for financial aid to determine the impact of the correction of misperceptions on take-up, and misperceptions will be elicited again. The results help us to understand how misperceptions about social benefit programs relate to non-take-up, how correcting potential misperceptions influences application rates, and which reasons for non-application are especially prone to be tackled by information.
External Link(s)

Registration Citation

Citation
Riedmiller, Sebastian. 2024. "’I Won’t Get it Anyway’ - A Field Study on Misperceptions About Financial Student Aid and Reasons for Non-take-up." AEA RCT Registry. August 21. https://doi.org/10.1257/rct.11249-5.0
Experimental Details

Interventions

Intervention(s)
Intervention:
Participants receive bundled information about the financial aid program. This intervention consists of parents' income thresholds, financial aid amounts, repayment modalities, own income and wealth, and age. Additionally, if the participant answered the questions about the parents' income, she receives additional feedback about the potential threshold of financial aid she could get per month. If the calculation of a positive threshold is not possible due to parents' income above the eligibility threshold, students are informed that a calculation was not possible based on their answers. Students who do not indicate the marital status of their parents or have one deceased / not available parent receive an income threshold for their parent(s) that would still result in a positive amount of financial aid for the student. All students receive links to information websites of the financial aid program and are invited to contact the research team via email if they want to receive more information or want to talk about the financial aid program.

Cross-randomized intervention:
All participants receive an email about the follow-up survey three months between the initial data collection and the recontact. A stratified subsample receives a text passage reminding them that our calculations have determined a positive financial aid amount, and that if one wants to receive financial aid for the next semester, one should file an application shortly. The subsample is cross-randomized from the initial treatment and control groups.
Intervention (Hidden)
Intervention Start Date
2023-05-02
Intervention End Date
2023-05-31

Primary Outcomes

Primary Outcomes (end points)
Application rate:
An indicator variable whether the individual student filed an application for the financial aid program between the first wave of data collection and the second.

Financial aid amount:
A continuous variable measuring how much financial aid the student receives in case the application was successful.
Primary Outcomes (explanation)
Application rate:
Six months after the first wave of data collection, students are contacted again and asked if they have filed an application for the financial aid program in the meantime. If yes, this variable is equal to 1, and 0 otherwise.

Financial aid amount:
If they indicate that they filed an application, students are additionally asked for the monthly amount of financial aid they receive. This is used as an outcome variable. This variable is continuous but truncated at 0€ and 934€ since 934€ is the upper bound of the financial aid one can receive.

Secondary Outcomes

Secondary Outcomes (end points)
Misperceptions about program criteria:
Based on three hypothetical scenarios, students are asked about their perceptions about the amount, parents' income thresholds, and repayment modalities of the financial aid program. Each question has an underlying correct answer. Deviations from this answer are seen as misperceptions.

Reasons for non-take-up:
Students are asked what are the reasons behind their decision not to have filed an application yet. Answers are given on a 5-point Likert scale ranging from "Applies" (1) to "Does not apply" (5).
Secondary Outcomes (explanation)
Misperceptions about program criteria:
Three hypothetical scenarios are used to elicit perceptions about the financial aid program's conditions, namely financial aid amounts, parents' income thresholds, and repayment modalities. In each scenario, the students are presented with simple cases that include all the information they need to give an answer. For the first scenario, students are asked about how much financial aid the person receives per month. In the second scenario, I ask them about parental income. Lastly, in the third scenario, students are asked how much money needs to be paid back. Additionally, students are asked what the fraction of students that received the financial aid in 2019 (pre-covid) is and how much they received on average per month. A deviation from the underlying correct value is considered a misperception.

Reasons for non-take-up:
If students indicate that they are not receiving the financial aid this year and did not file an application yet, they receive a likert matrix with possible reasons why they did not apply. Answers are given on a 5-point Likert scale ranging from "Applies" (1) to "Does not apply" (5).

Experimental Design

Experimental Design
I conduct an online experiment with German university students. The experiment is conducted with oTree. Students are invited to participate through their general student committee distribution channels, varying from email distribution lists, newsletters, and/or social media posts. The survey consists of three parts: Socio-demographic background questions, elicitation of perceptions, and reasons for (non-)take-up of the financial aid.

Socio-demographic questions:
Students are asked how they finance their monthly living expenses. For each possible income source, they have to type in an amount how much money they have from this source per month. Depending on whether they indicate to already receive the financial aid by entering a positive amount in its respective input field, students receive additional questions to determine their history with the financial aid program. On the next page, students are asked about their parents' marital status, job status, monthly net income and confidence in these values, and dependent siblings to determine potential eligibility for the financial aid program. Additionally, students are asked about their year of birth, sex, housing situation, and current educational status.

Elicitation of perceptions:
Students receive 10 questions about the conditions of the financial aid program. The first 8 questions are elicited with hypothetical scenarios. These scenarios describe a situation of a student that receives the financial aid.
In the first scenario, students have to give their perception about how much financial aid the respective person from the scenario receives per month. This question is asked three times based on the scenario and two changes in the scenario. These changes are an adjusted parents' income and an adjusted wealth, which both translate into a change in the amount of financial aid one receives per month. The perception elicitation is therefore repeated to measure how changes in the situation translate into adjustments of perceptions. An answer is correct if it is in the 200€-interval around the true value.
In the second scenario, the participant receives information about how much financial aid the person in the hypothetical scenario receives per month and is asked about the parents' income. This question is repeated a second time after adjusting the number of siblings in the scenario, which changes the parents' income amount for receiving financial aid. An answer is correct if it is in the 15,000€-interval around the true value.
The third scenario gives information about the total amount of financial aid the hypothetical student received over time. The participant is asked about how much this person has to pay back under three different circumstances, each of which results in a different repayment amount. An answer is correct if it is in the 1,000€ interval around the true value.
Additionally to these scenarios, the participant is asked about the fraction of students that received the financial aid in 2019, where an answer is correct if it is in the 10%-interval around the true value, and the average financial aid these students received per month, where an answer is correct if it is in the 200€-interval around the true value.
These 10 questions are incentivized by increasing the individual chance to win a lottery of 100 x 25€ by 10 percentage points per correct answer. For each perception question, participants are asked about their confidence in their answers.
In addition to the incentivized elicitation of perceptions, participants who did not file an application for the financial aid yet are asked if they think they would be eligible on a 5-point Likert scale.

Reasons for (non-)take-up:
Depending on whether the participant already receives the financial aid or applied for it, she is asked for the reasons for take-up using a 5-point Likert scale matrix. Participants that never filed an application are asked for reasons for non-take-up, also using a 5-point Likert scale matrix.

Intervention:
Participants receive bundled information about the financial aid program. This intervention consists of parents' income thresholds, financial aid amounts, repayment modalities, own income and wealth, and age. Additionally, if the participant answered the questions about the parents' income, she receives additional feedback about the potential threshold of financial aid she could get per month. If the calculation of a positive threshold is not possible due to parents' income above the eligibility threshold, students are informed that a calculation was not possible based on their answers. Students who do not indicate the marital status of their parents or have one deceased / not available parent receive an income threshold for their parent(s) that would still result in a positive amount of financial aid for the student. All students receive links to information websites of the financial aid program and are invited to contact the research team via email if they want to receive more information or want to talk about the financial aid program.

Cross-randomized intervention:
All participants receive an email about the follow-up survey three months between the initial data collection and the recontact. A stratified subsample receives a text passage reminding them that our calculations have determined a positive financial aid amount, and that if one wants to receive financial aid for the next semester, one should file an application shortly. The subsample is cross-randomized from the initial treatment and control groups.

Recontact:
Students that indicated to have never applied for the financial aid in the past are recontacted again six months after the first survey to take part in another follow-up survey. Now, students are asked if they have applied for the financial aid program in the meantime and if so, which amount they receive per month. Misperceptions are elicited again through different scenarios. Additionally, I elicit expected income at age 45, expected or actual time taken to file application depending on application status, debt aversion impulsiveness, patience, migration background, parents' educational background, GPA, and current enrollment status. Participants from the intervention group who file an application are also asked which part of the intervention got them to file an application with a 5-point Likert scale matrix. Participation is incentivized by a lottery to receive 200 x 50€ for participation.
Edit after data collection: Since 132 students indicated to have applied for financial aid but do not know the final outcome, I recontact the students again 12 months after the intervention. I additionally ask for the students’ income, their housing situation, GPA, and enrollment status again. Participation is again incentivized by a lottery to receive 200 x 50€. Additional incentives are used to collect the data of students among the 132 participants with pending applications who do not participate otherwise.

Hypotheses:
(i) Stronger misperceptions lead to a lower likelihood to file an application.
(ii) Students who receive the intervention are more likely to file an application before the recontact.
(iii) Students have heterogenous treatment effects with respect to their stated reasons for non-application.
Experimental Design Details
Randomization Method
Randomization is done with the minMSE R-package. Randomization is stratified based on the university, course of study, and cohort. Additionally, the federal state of the university, the number of students, university specialization, and the distributional channel of the respective general student committees are taken into account for treatment assignment.
Treatment assignment took place separately for student committees that indicated to distribute the participation link and those who did not.
Randomization Unit
Randomization takes place at the cohort level. Treatment assignment is stratified by university, course of study, and year of study.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
In total, there are 83 universities that were contacted. 50 of them answered to distribute the participation link in some form. Each university is considered with 18 courses of study and 7 cohorts per course of study, leaving us with 6300 clusters. Each cluster corresponds to a specific cohort of a specific course of study at a specific university, respectively. Therefore, depending on participation rates, some clusters possibly stay empty of observations.
Sample size: planned number of observations
Assuming that on average 2% of the students of the 50 universities that distribute the participation link take part, the number of observations is 21,620 students. Of those, 44% fulfill the general eligibility requirements of the financial aid and do not already receive it. Assuming a recontact rate of 50%, we end up with 4750 students.
Sample size (or number of clusters) by treatment arms
2375 students control, 2375 students treatment.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Without assuming restrictions, the MDES is 0.085 standard deviations with 80% power and a significance level of 5%. Assuming that 10% of the sample file an application, this corresponds to a treatment effect of 2.55 percentage points. Using a more conservative approach assuming an effect size variability of 0.1, the MDES is 0.154 standard deviations, which corresponds to a treatment effect of 4.62pp.
IRB

Institutional Review Boards (IRBs)

IRB Name
Ethics Committee of the Faculty of Economic and Social Sciences (ERC-FMES) at the University of Cologne
IRB Approval Date
2023-03-03
IRB Approval Number
230011SR

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
May 31, 2023, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
December 15, 2023, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
3166 clusters are filled after the first wave. After the second wave and only considering the students who fulfill the general eligibility requirements of the financial aid and do not already receive it, I end up with 1977 clusters.
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
22,222 students finish the first survey in May of which 17,636 consent to be contacted for the second wave. 12,636 students participate again of which 6,501 are part of the experiment. The total response rate is 68.59%. The response rate for the experiment is 67.21%.
Final Sample Size (or Number of Clusters) by Treatment Arms
3,399 students from the control group participated in the second survey and 3,102 from the treatment group.
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials