Local Language Acquisition and Social Integration

Last registered on October 31, 2024

Pre-Trial

Trial Information

General Information

Title
Local Language Acquisition and Social Integration
RCT ID
AEARCTR-0012827
Initial registration date
October 20, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 28, 2024, 12:52 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
October 31, 2024, 12:54 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
University of Stavanger Business School

Other Primary Investigator(s)

PI Affiliation
Corvinus University of Budapest
PI Affiliation
University of Stavanger

Additional Trial Information

Status
In development
Start date
2024-10-15
End date
2026-01-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Learning the local language is often recommended to foreigners to socially integrate into a new country. We study the extent to which acquiring the local language creates social opportunities for foreigners. Using a field experiment, we investigate the relationship between cultural assimilation via learning the local language and non-native individuals’ ability to socially integrate along two dimensions: (1) housing, and (2) amateur football clubs. We use online messages to contact landlords/realtors to view a property and ask football coaches to join a trial practice. We compare response rates across different email treatments.
External Link(s)

Registration Citation

Citation
Nesseler, Cornel, Hammad Shaikh and Jiaqi Zou. 2024. "Local Language Acquisition and Social Integration." AEA RCT Registry. October 31. https://doi.org/10.1257/rct.12827-1.1
Experimental Details

Interventions

Intervention(s)
We examine the degree to which acquiring proficiency in the local language can increase social opportunities for foreigners. We will send thousands of requests to join an amateur football club and view a rental property using a foreign-sounding name, randomly varying whether the message is sent in proficient English, broken local language (A1/A2), or proficient local language. Additionally, when English or a broken local language is used, we will have an additional treatment condition which will mention in a subtle way that the person is taking a local language class. We will also have a control group where the message is sent in the native language using a native-sounding name. We will then compare response rates by football coaches and landlords/realtors across the different message variations. Such an experimental design will enable us to study whether signaling efforts to learn the local language and acquire some degree of proficiency in the local language will increase social opportunity in joining a football club and finding a place to live.
Intervention Start Date
2024-10-21
Intervention End Date
2025-08-31

Primary Outcomes

Primary Outcomes (end points)
The primary outcomes are as follows:

1) indicator variable for whether the football coach responds to the email,
2) indicator variable for whether the realtor or landlord (listing poster) responds to the email

Primary Outcomes (explanation)
The outcome variable is coded as 1 if there is any type of response and is 0 otherwise. We wait at least 3 business days to allow sufficient time for the realtor/landlord or coach before recording the response.

Secondary Outcomes

Secondary Outcomes (end points)

The secondary outcomes are as follows:
1) indicator for opening the email when observed
2) indicator variable for whether the response is unconditionally positive (yes),
3) indicator variable for whether the response is making further inquiries (yes, but...),
4) indicator variable for whether the response has a positive sentiment (yes or yes, but...)
5) indicator variable for whether the response is negative (no),
6) indicator variable for whether the response is received within 24 hours
Secondary Outcomes (explanation)
1) We can observe a measure of whether the email is opened when the emails are deployed using an email plugin. This is 1 if the person opened the email, and 0 otherwise. We may not have this variable for Austria and Hungary as we may not be able to use email plugin in those countries, but rather contact forms associated with the posted listings.

2) The response is positive when the respondent welcomes us to come for practice or see the property unconditionally on other factors. For example, "The property is available, I can show you on Tuesday at 5 pm" or "You can come for practice this Friday at 6 pm" would be an unconditional positive response. We code this as "Yes"

3) We further distinguish between a positive response and a response with further inquiries. For example, for football coaches: "Where did you play before?" or for landlords/realtors: "Where did you find this advertisement?". We code this as a "Yes, but ..." Such inquiries are coded as 1 and are 0 otherwise for the further inquiries indicator variable.

4) We will aggregate our "yes" and "yes, but ..." classification into one positive response variable. We may not have a lot of variation in the data between secondary outcomes 2) and 3), so this aggregating would be more informative.

5) For whether the response is negative, a value of 1 is assigned to the responses where the email sender is in any way denied permission to join for trial practice or denied viewing of a property and is 0 otherwise. For example, "The property is not available" is coded as "No"

6) We will compare the email sent date to the reply date to determine whether a response was delivered within 24 hours

Experimental Design

Experimental Design
The study takes place across 4 countries in central Europe: Austria, Hungary, Czech Republic, and Slovakia. We send emails to amateur football clubs asking them to join for trial practice and send requests to view a rental property. To assess whether taking efforts to learn the local language as a foreigner increases opportunities to socially integrate across these two dimensions, we compare response rates across various text variations.

We will have 6 different message manipulations: (1) local name and proficient local language, (2) foreign name and proficient local language, (3) foreign name, broken local language, and taking local language class, (4) foreign name and broken local language, (5) foreign name, English, and taking a local language class, (5) foreign name, English, and taking a local language class, and (6) foreign name and English. The control group is the text version where a native-sounding name and the native language are used, remaining are 5 different treatment conditions that use a foreign-sounding name.

The messages that employ the broken local language were constructed after consulting online language teachers, native speakers of the language, and chatGPT. We first wrote both the broken and proficient messages in English and then had online language teachers translate the messages into German, Czech, Slovak, and Hungarian. Even for writing the messages in broken English, we consulted an online English language teacher.
Experimental Design Details
Not available
Randomization Method
We carry out randomization in Stata to assign football clubs and rental properties to the 6 text variations in a way so that we have balance in the number of observations across each manipulation within each strata. Randomization is stratified at the country-region level. For example for rental listings we intend to using cities as that is readily available on the ad. For football, we will use county or district. We will combine small regions together into its own Strata. Additionally we will send the emails over Mon - Friday day of the week, so will include day of week as an another strata. That is the strata will be at the region-day level. The random assignment of observations to treatment will be replicable using the associated seed number. We intend to use the stratarand command in Stata to carry out the randomization.

For countries with a large sample size, we may employ two different organizational accounts for the deployments to limit the potential of being detected as spam. In this case, we will also randomize over observations assigned to the organizational accounts.
Randomization Unit
The unit of randomization is a football club or an advertisement for a rental property listing. The randomization is carried out within a day of week (message send day) - country-region strata.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
The randomization is carried out within day-country-region Stratas. We plan to have around 5 x 4 x 5 = 100 strata, with several 100s of observations per strata.
Sample size: planned number of observations
We expect to have between 2500 - 4000 football clubs and between 6000 - 10000 rental property advertisements.
Sample size (or number of clusters) by treatment arms
As we have 6 text versions, we expect to have between 400 - 650 observations per treatment arm for the football experiment, and at least 1000 observations per treatment for the housing experiment.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We will compute the MDES using the PowerUp software provided by Dong, N. and Maynard, R. A. (2013). We will compute the MDES to compare the control group (native, proficient local language) to one of the 5 treatment conditions as these are our primary comparisons of interest in our study. Assuming we have 40 blocks (regions) and around 21 observations per block, and assuming constant treatment effects, a significance level of 5, two-tailed test, partial R^2 for the blocks as 0.1, and setting the power to 80%, we achieve an MDES of 0.18 SD. Further including covariates such that the R^2 increases to 0.2 decreases the MDES to 0.17 SD. Assuming an average response rate of 40%, the MDES for the response rate differential is around 8 percentage points when comparing across two groups. Increasing the sample size to a plausible 4000 observations decreases the MDES to 0.14 SD or 7 percentage points. We expect the power to be larger in the housing experiment than the football experiment due to a larger sample size. Our sample size in the housing experiment may be sufficiently large to have a MDES of 0.1 SD or around 5 percentage points. These effect sizes are aligned with prior literature which typically finds large effects when comparing the response rate of football coaches or the callback rate of hiring managers across foreign-sounding names and native-sounding names.
IRB

Institutional Review Boards (IRBs)

IRB Name
Corvinus University of Budapest Office of the Vice-Rector for Research - Research Ethics Committee
IRB Approval Date
2024-01-15
IRB Approval Number
KRH/16/2024
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information