Randomized Educational Interventions on Social Interactions and Incentives in the Classroom Environment

Intervention 1 (Cooperation): This intervention aims to induce cooperative interactions among students through teamwork, within a novel program where children learn how to code.

Intervention 2 (Competition): This intervention induces a learning/performance environment that involves competitive elements, again within a novel coding program.

Intervention 3 (Individual Coding): In this intervention, children learn coding individually, without structured cooperative or competitive interactions/incentives.

Pure Control: This is a group of schools where no external program is implemented (the policy-relevant control group in the Turkish educational setting).

The interventions are implemented in-class, by external instructors trained extensively by the research team, in a large sample of 4th graders in public elementary schools in Istanbul, Turkey.

The goal of the interventions is to create a novel educational program on coding, and use it as a backdrop for inducing certain types of social interactions. An educational context is necessary for creating an environment that can mimic the classroom, with exogenous rules for interactions, performance, and rewards. To this end, we create our own, controlled educational setting where interactions are induced in pre-determined ways and there is a clear performance context with predetermined rewards. Notice that the general RCT embeds three different interventions. The cooperative learning intervention targets outcomes distinct from the intervention that involves competitive elements. Containing the same coding curriculum but stripped of competitive and cooperative elements, the individual coding intervention both allows us to study the research question of the impact of teaching children coding, and also serves as a control for the other two interventions.

In order to be able to keep track of children's interactions, performance and learning, we design our own digital platform for teaching coding. This platform hosts a set of games that we modify and adapt from widely-used programs to teach children coding, as well as novel games that are designed to induce specific types of interactions between children within the coding context. We supplement these digital games with several pencil-and-paper activities. Intervention content, rules and rewards are meticulously designed by the research team for each week.

As mentioned above, we have three coding curricula: cooperative, competitive, and individual. In addition, we have a pure control group. In the first two treatments, randomly formed pairs of students are seated next to each other and share a tablet, while in the individual treatment, children work on their own tablets individually, but are still randomly paired in terms of seating arrangement. In the educational context where the program is implemented, children always sit in pairs. It is therefore difficult to create a situation where the coding program is given individually, in total social isolation. However, we minimize social interaction within the individual treatment by making sure that children work individually and by instructing children to not communicate during the games. In this treatment, we also pair children randomly, to be able to isolate the effect of induced interactions from just sitting together (in order to have more refugee-host student pairs for research questions related to exposure to outgroups, we exclude refugee-refugee pairings). The pure control group does not get any program, and serves as a control for the three coding treatments. In order to be able to compare learning outcomes in the different types of curricula, we make sure to have the same amount and type of material in the different treatments, to the extent possible.

We work with 21 externally hired instructors, who visit each classroom for two consecutive class hours in a week to give the program. The program is expected to run for 11 weeks (excluding two planned breaks). Instructors are extensively trained by the project team before they are sent to the classroom. To minimize potential instructor effects, we specify in detail everything to be done during a class hour, in detailed instructor manuals and in training. All main content is delivered through videos prepared by the research team, and the role of the instructor is mostly relegated to enabler and guide. Still, we rotate the teachers to make sure that children are exposed to different instructors.

2022-09-26

2022-12-31

The larger RCT embeds three different interventions. The outcome variables hypothesized to be affected partially differ for the three different interventions.

In terms of methodology, outcome variables to be collected can be grouped under the following categories: 1) Outcomes measured by incentivized behavioral tasks, 2) Outcomes measured by self-reports, 3) Outcomes measured by objective tests administered by the research team, 4) Administrative data (e.g. school grades, behavioral conduct), 5) Data coming from teacher surveys.

Below are the primary outcomes associated with each intervention.

Intervention 1 (Cooperation): Primary outcome measures are cooperative behavior, in-group and out-group altruism, beliefs about/attitudes toward out-groups (over gender and refugee status), friendship networks, willingness to compete and gender differences therein, classroom cohesion level, behavioral conduct at school, learning outcomes from the coding program, performance in math, behavior in groups (leadership, power).

Intervention 2 (Competition): Primary outcome measures are willingness to compete and gender differences therein, beliefs about/attitudes towards out-groups, friendship networks (over gender), learning outcomes from the coding program, performance in math, gender-related behavior in groups (leadership, power).

Intervention 3 (Individual Coding): Cognitive skills, algorithmic thinking, strategic thinking, performance in math, creativity, self-confidence in math, occupational/STEM aspirations, learning outcomes from the coding program.

Details of these measurements, several of which will involve the design of novel behavioral tasks, are to be provided in the pre-analysis plans for each intervention.

Note that the measures described for each intervention are to be collected in all four groups, since the two distinct interventions (coding with cooperation and competition) share the individual coding and pure control groups as control. However, the interventions target different types of behavior, and hence give rise to partly different hypotheses in terms of expected effects on behavior (described in the pre-analysis plans). For example, while all coding interventions are hypothesized to improve algorithmic thinking and STEM aspirations, only the cooperative intervention is hypothesized to improve prosociality.

Data on some variables will be collected to put forward data for mechanisms for treatment effects and for heterogeneity. For the intervention with competition, these include self-confidence, social confidence, survey measures of attitudes towards and response to competition (e.g. how relaxed one is in competitive environments), and a behavioral measure of risk attitude. For the intervention on teamwork, these include self-confidence, empathy, social confidence, how well the teams interacted/worked together, and attitudes towards responsibility. The pass-through of attitudes and peer effects (e.g. on competitiveness and grit) across the different types of curricula will be studied as a separate research question, and related variables will be collected (to be described in pre-analysis plan).

We have 35 schools in each treatment cell: competition, cooperation, and individual, and 35 in control; a total of 140 schools. In each school, we have three classrooms in the program, randomly selected out of the classrooms that satisfy pre-established criteria as regards class size and the number of refugee students. Every classroom in the same school receives the same treatment (cooperation, competition, or individual coding). Baseline data were collected from the schools in our sample in May-June 2022.

Our sample comes from districts in Istanbul where the number of refugees is large. In order to have enough power to answer research questions related to attitudes towards refugees, we focus on classrooms that had between 27 and 44 students registered, with at least four refugee students in the class roster (as per the information we received at the time of planning the intervention) but where refugees were not the majority in the classroom. We take schools that have at least three classrooms that satisfied the above criteria, in districts that are not in the far outskirts of Istanbul (for logistic reasons) as our school sample. In schools that had more than three classrooms fitting the criteria, we randomly selected three and visited those. We have 35 schools in each treatment cell: competition, cooperation, and individual, and 35 in control; a total of 140 schools. Every classroom in the same school receives the same treatment (coding with cooperation, coding with competition, or individual coding).

In May-June 2022, we collected detailed baseline data from these schools. Endline data collection is planned for three phases: January 2023, after the intervention, and again in February 2023, and in May-June 2023. In addition to data collected in our measurement visits, we will be using rich within-intervention data collected through our coding app to disentangle mechanisms of treatment effects and study several auxiliary research questions. A longer-term data collection is planned for after the students move on to middle school, in Spring 2024. This is conditional on permissions and depending on the extent of data collection that will be possible, a supplemental registration will be made for this phase.

Randomization was done in office by a computer.

The unit of randomization is the school.

Yes

140 schools.

The number of students officially registered in the rosters received at the time of study plan was 14512.

35 schools in coding with cooperation, 35 schools in coding with competition, 35 schools in coding-individual, 35 schools in pure control.