Principals and minorities: a field experiment

2021-03-23

2021-03-26

The primary outcome will be a binary variable (Yes, No) for receiving a response to the query.

Second, we will measure the time to response and length of response. Third, we will measure the courteousness and informativeness of the responses.

Courteousness and informativeness will be measured using two independent evaluators. The scale will have three levels (-1, 0, 1), zero for neutral, one for courteous/informative, minus one for non-informative/non-courteous. If the evaluators disagree on how to code a particular response, this will be flagged and evaluated by the researchers.

This is a field-experimental study.

Background and motivation

There is a significant amount of observational and anecdotal evidence documenting systemic discrimination of Roma in Czech schools and their adverse educational outcomes. Differential treatment by the school principals, if any, may thus reinforce the repercussions of existing discrimination against Roma. At the same time, Roma children typically come from already disadvantaged backgrounds, which may be partly responsible for the adversity of their educational outcomes (for further background and references see Montag and Mikula, 2021, linked in the Analysis plan). We further note that the treatment and performance of other minorities' children are not well understood and are generally under-researched, yet it is of similar concern.

This field experiment, therefore, tests for the presence of differential treatment of minorities in elementary school admission in the Czech Republic. Our focus is on four significant minorities in the country: Roma, Slovak, Ukrainian, and Vietnamese. The study is designed to tap into, and identify the presence of, the two key mechanisms that may drive discrimination: ethnic animus and socioeconomic status. In addition, we test for differential treatment of boys and girls, and whether and how it varies with ethnicity.

We manipulate three variables of interest: (i) The putative ethnicity of the sender using names with ethnic connotation. (ii) The socioeconomic status (integration level) of the sender by varying the grammatical quality of the queries. (iii) The gender of the child stated in the queries.

1. Names and ethnicity signals (Czech, Roma, Slovak, Ukrainian, Vietnamese)

Because of legal constraints, neither the Census nor any administrative dataset containing ethnicity and names is available in the Czech Republic. Furthermore, our sender personas are female, whereas the previous correspondence experiments in the Czech context use fictitious personas who are male (Bartos et. al 2016 and Mikula and Montag 2021).

We have therefore selected female names using several sources and steps. We started with surnames in the appendix of Bartos et. al (2016, Czech, Roma, and Vietnamese names) and Mikula and Montag (2021, Czech and Roma names). We have selected common-sounding Vietnamese and Ukrainian names using Wikipedia and Google search. We have then tested the ethnicity signals associated with these names in an in-class student survey (n = 71, Economics 101 class at Charles University). The students were asked to select the most likely ethnicity for 16 names (four per ethnicity, randomly ordered). The options of ethnicities to choose from were Czech, Slovak, Roma, Ukrainian, Russian, and Vietnamese.

Based on this first survey, we selected two Czech and Vietnamese names (Jiřina Hájková, Věra Svobodová, Giang Uyen Pham, and Mai Li Nguyen). More than 98 % of students stated that these names are most likely Czech and Vietnamese, respectively.

The two names most frequently, 65 % and 77 %, respectively, associated with Ukrainian ethnicity were Anna Shevchenko, Yelyzaveta Tkachenko. We note, that 32 % and 17 % of students, respectively, associated these two names with Russian ethnicity. However, the exact distinction between these two ethnicities is of secondary importance for us. Hence we use these two names as putatively Ukrainian (or mixed Ukrainian and Russian).

The ethnicity signal of Roma names turned out to be mixed with Slovak ethnicity. For the two Roma names most frequently associated with Roma ethnicity (Silvie Gažiová and Klaudie Lakatošová), 49 % and 50 % of students stated they are Roma, respectively. 41 % and 28 %, respectively, stated these are most likely Slovak.

Because of this inconsistency, and because identifying the differential treatment of Roma is of key importance to us, we have decided to include Slovak ethnicity in our study. This allows us to separate the Roma-specific signal and the Slovak "component" (see the Analysis plan for details).

Thus, we have selected four Slovak-sounding female names using Google search and Wikipedia. Because there is an overlap between typical Czech and Slovak names, selected Slovak surnames that have typical Slovak diacritics (Ľuptáková), Slovak first names that have Slovak spelling (e.g. Katarína), and Czech first names with Czech diacritics (Jiřina).

After this, we have set up another (out-of-class survey) in which 582 students (Economics class at Masaryk University) were presented with 13 names, four Czech, four Slovak, and five Roma, randomly ordered, and asked again to assign the most likely ethnicity of these names. The choice was between Czech, Slovak, Roma, Hungarian, Ukrainian, and Russian. We have received 294 responses (response rate 50.5 %, 194 respondents self-identified as Czechs, 79 as Slovaks, 19 as other, one declined to self-identify).

Focusing on answers from Czech respondents, over 99 % of students assigned Czech ethnicity to the two putatively Czech names (Jiřina Hájková, Věra Svobodová). The two putatively Slovak names most frequently associated with Slovak ethnicity were Ľudmila Húsková and Katarína Ľuptáková, with 90 % and 87 % of Czech students assigning Slovak ethnicity to them (the second most frequent choice was Ukrainian, with 5 % and 7 % frequency respectively).

For putatively Roma names, three turned out to provide a similar signal of Roma ethnicity: Klaudie Lakatošová, Karmen Horvátová, and Silvie Gažiová, 57 %, 52 %, and 54 %, of students, respectively, stated these names most likely belong to Roma persons. The second most frequent choices were Czech or Slovak (18 %, 15 %, and 18 %). We have selected the first two names, because of the higher proportion of respondents perceiving these as Czech (18 % and 12 %, vs 6 %), which is empirically the least problematic noise to address in our estimation (see the Analysis plan for details).

2. Queries

April is the enrollment period, during which individual schools organize an official (but non-mandatory) enrollment day in which parents and pupils visit schools and formally enroll. Our goal is to minimize the costs that our queries generate to the school principals. We, therefore, send out a simple two-sentence query stating that the sender (mother) is considering to which school her child should be enrolled and asking whether there will be an open day or an online information meeting for parents. To this query, the principal may simply state "Yes" or "No" and in the former case give a day and time or a link.

3. Literacy signals our queries will signal the parent's socioeconomic status (and/or integration level) via grammatical quality of the queries. In order to obtain low-literacy queries, we have asked several participants in a Czech as foreign language course to draft the queries for us. Grammatically correct queries will be drafted by us. We will send out two versions of the query per literacy signal (identical content but different wording).

4. We will also manipulate the child's gender by simply specifying that the mother wishes to enroll her boy/girl.

Computer (R version 4.0.4 functions "sample()", package base, or "slice_sample()", package dplyr, set.seed(42).

School.

No

Not applicable.

The sample consists of the universe of schools in the Czech Republic, 4241 schools altogether, taken from a database we purchased for this project. We drop schools with missing or non-unique email contact. If the same person is principal at multiple schools, we randomly select only one of them. This yields us a final sample of 4113 schools (school principals).

We have 5 ethnicities x 2 signals of socioeconomic status x 2 genders of the enrolling child = 20 treatment arms, each with 411 or 412 units.

We note, that, in order to minimize the costs to the principals during the potentially busy enrollment period, we send out only one query per school/principal, relying on between-subject variation. Our main hypotheses test for differential treatment (ethnic discrimination) of minorities versus the Czech majority measured by the response rates. We use the exact unconditional z-test (Suissa and Shuster 1985). In order to account for multiple hypotheses, we have computed the power for alpha = 0.01. We consider a four percentage points difference in response rates as a substantively meaningful effect, similar in magnitude to the estimates in Gulietti et. al (2017). Assuming, conservatively, the response rates of about 0.5, and thus the maximal possible variance of 0.25, our samples of 822 observations per ethnicity yield a power of 0.82 for identifying a four p.p. difference between any two ethnicities in our data.