Eliciting Attitudes in Statistics Course

The intervention involves presenting students with certain exam preparation questions during 3 tutorial sessions of a statistics course in order to elicit preferences.

This experiment relates to a larger evaluation of a diversity-related policy of randomized working groups at a large European university. Rather than allowing students to choose their own groups for academic group work, the university introduced a policy of random groups: students are (unconditionally) randomized into working groups during 3 blocks of the academic year for group work. The aim of this experiment is to establish whether exposure to minorities (in terms of ethnicity, nationality, and gender) during this group work altered majority students' attitudes towards such minorities.

To elicit these attitudes, our experiment will be implemented as part of a mini-exam questions in each of the 3 tutorials belonging to a statistics course. We refer to them as the experimental questions. In these experimental questions, students will be presented with a question based on a real assignment from a closely related course. In addition, they are given three potential answers to the question. Students will be then asked to pick (i) the best of 3 given answers, and (ii) the worst of them. The experimental questions will, for instance, relate to the validity of an instrumental variable, the interpretation of a coefficient, or the (economic) relevance of certain results. The differences in the quality of the answers will be small, although an objective ranking will always exist. There will be 1 experimental question per tutorial session.

The 3 potential answers provided in the experimental question will be based on real answers from previous students, and this will be made clear to the participants. In addition, the (fictitious) names of these previous students will be deliberately revealed. Some names will belong to minority students (female, student with migrant background, international student). These names will be generated by the researchers and will not belong to the original student who answered the question in the related course, nor to any real student at the university, past or present. The names will be randomly allocated to different answers across tutorial groups, in order to control for answer quality and so separately identify any biases in assessment.

We will study whether (i) previous answers from minority students are systemically ranked lower, and if so, (ii) whether exposure to more minority students decreases this bias. Our dependent variable will be a function of the answer ranking (i.e. a variable indicating the rank given to the answer belonging to the fake minority student), and our independent variable will be some function of the number of minority students in the individual's working groups (e.g. the proportion of minority students in student's working groups).

2020-03-17

2020-04-17

Probability of picking answer from minority student (in terms of gender, ethnicity, or nationality) as (i) the best answer, and (ii) as the worst answer.

The experiment will take place during 3 tutorial sessions of a statistics course.

As part of a diversity policy, students have been allocated to random working groups (of 2 to 5) students in the previous year. We exploit this random variation to study how the exposure to minority students changes attitudes towards minorities. Some majority students will have (randomly) been exposed to no minority students, while some to many minority students.

To study the effect of the intensity of this exposure, this experiment will elicit such attitudes during 3 tutorial sessions of a statistics course. Namely each student will be randomly assigned a different mini-exam during each of the 3 tutorial sessions. The order in which the names of the fictitious previous students appear will be randomized across each TA - each of which teach 1-3 tutorials consisting of an average of 35 students each. We will exploit this random variation in the position of the previous student's name in order to elicit attitudes that each current student has about minorities.

Randomization done in office by a computer.

The randomization of mini-exams (and hence the order of the fictitious names) will be done at the level of the teaching assistant for the tutorials.

Yes

1 course.

700 students.

As the number of tutorials that each TA teaches varies, the sample size per treatment arm also varies. The 9 TAs teach between 1-3 tutorials each per session. Thus, the sample size per treatment arm varies from approximately 35 students to 105 students.