Intervention (Hidden)
Students of a Coursera Contracts course will be randomly assigned to one of the five groups. In each group, the substantive content will remain the same – same words spoken and same words shown in powerpoint slides and photos, but the groups will differ with the visual image and voice of the professor. Here is a description of the specific groups:
• Control group: Students in this group will learn the course material by being shown the preexisting videos with the professor delivering the material throughout the course
• Treatment group 1: Students in this group will learn the course by being shown videos in which the professor is replaced by a black male avatar instructor.
• Treatment group 2: Students in this group will learn the course by being shown videos in which the professor is replaced by a black female avatar instructor.
• Treatment group 3: Students in this group will learn the course by being shown videos in which the professor is replaced by a white male avatar instructor.
• Treatment group 4: Students in this group will learn the course by being shown videos in which the professor is replaced by a white female avatar instructor.
The following list presents specific questions of interest along with the appropriate statistical approaches to studying them:
1. Do students perform better or have better engagement when they are taught by avatar instructors than when they are taught by the professor?
Method 1: OLS regression examining the effect of the type of instructor (whether they were taught by any one of the avatars or by the real professor) on students’ overall grade (or the number of items passed)
(1) Without student controls
• Dependent variable: students’ overall grade (or any of the other outcome variables)
• Independent variable: indicator if they were taught by an avatar instructor
(2) With student demographic (age, race, gender, level of education) controls
Method 2: OLS regression examining the effect of the specific type of instructor (black male avatar, black female avatar, white male avatar, white female avatar, or real professor) on students’ overall grade (or the number of items passed). This method allows us to see a more detailed effect of each of the treatment groups on the student outcome.
(1) Without student controls
• Dependent variable: students’ overall grade (or any of the other outcome variables)
• Independent variables:
- indicators for each of the four avatar types (black male, black female, white male, white female)
(2) With student demographic (age, race, gender, level of education) controls
(3) To observe heterogeneous treatment effects:
• Dependent variable: students’ overall grade (or any of the outcome variables)
• Independent variables:
- indicators for each of the four avatar types (black male, black female, white male, white female)
- Uninteracted student demographic indicators (age, race, gender, level of education categories)
- Interaction of the avatar indicators with the student demographic category indicators
*** We will use interact coefficients to test whether particular types of students had superior outcomes with particular avatars
(4) Perform post-estimation (f-test) to see if each treatment effect various across different subgroups of student characteristics.
*** Significant coefficients here would indicate that the effect of instructor type differs across different student characteristic groups when it comes to students’ overall grades (or the number of items passed) as a measure of performance
3. Do students who are taught by an avatar instructor of the same race or gender as them perform better or have better engagement outcomes than those who are taught by an avatar instructor of a different race or gender?
Method 1: Regression examining the effect of students’ race (or gender) and the instructor’s race (or gender) on students’ overall grades (or the number of items passed), a measure of performance
(1) Without student controls:
• Dependent variable: students’ overall grade (or any of the outcome variables)
• Independent variables:
- Student’s race (gender)
- Instructor’s race (gender)
- Interaction of the student race and instructor race
(2) With student controls. This model also accounts for the heterogeneous treatment effects.
• Dependent variable: students’ overall grade (or any of the outcome variables)
• Independent variables:
- Student’s race (gender)
- Instructor’s race (gender)
- Interaction of the student race (gender) and instructor race (gender)
- Student characteristics (age, gender/race, level of education)
(3) Perform post-estimation (f-test) to see if each treatment effect various across different subgroups of student characteristics.
*** Significant coefficients here would indicate that the effect of instructor type differs across different student characteristic groups when it comes to each measure of performance
Method 2: Regression examining the effect of students’ race (or gender) and the instructor’s race (or gender) on students’ number of lessons completed (or whether the student completed their learning goal), a measure of engagement
(1) Without student controls:
• Dependent variable: the number of lessons completed (or whether the student completed their learning goal)
• Independent variables:
- Student’s race (gender)
- Instructor’s race (gender)
- Interaction of the student race and instructor race
(2) With student controls. This model also accounts for the heterogeneous treatment effects.
• Dependent variable: the number of lessons completed (or whether the student completed their learning goal)
• Independent variables:
- Student’s race (gender)
- Instructor’s race (gender)
- Interaction of the student race (gender) and instructor race (gender)
- Student characteristics (age, gender/race, level of education)
(3) Perform post-estimation (f-test) to see if each treatment effect various across different subgroups of student characteristics.
*** Significant coefficients here would indicate that the effect of instructor type differs across different student characteristic groups when it comes to each measure of engagement