Experimental Design
[Note: due to a tight timeline before school exams, we did not have time to pilot our endline survey on out-of-sample schools. In addition, we wanted to confirm the presence of a large “first stage” before deciding to roll out the full endline (since without a first stage, the endline would be a waste of money). As a result, we posted our pre-registration after 3 days of surveying were complete (~10% of schools), at which point we were confident in a reasonable first stage.]
We worked with a local NGO to run a school-level RCT in Tirunelveli district, Tamil Nadu. We worked with a sample of 176 secondary schools, randomizing half of those schools to implement a “House System,” with children randomized to one of four Houses, and school activities using the Houses for grouping students. We think of this intervention as a re-categorization-based approach (from social psychology), with the new group identity marker of “Houses” potentially reshaping the nature of pre-existing group identities, and with possible effects on broader student behaviors. The intervention began during the 2024/25 school year, and to save on cost, we did not administer a baseline survey.
We are now launching an endline survey of students. We are aiming to survey a random sample of 20 students per school, evenly spread across genders and school grades.
Some of the survey questions will help us establish a “first stage” – checking that the House System is both more common, and more intensively implemented, in treatment schools than in control schools. From some implementation monitoring and our initial endline surveys, we already know that our first stage is not one. As a result, we will focus on intent-to-treat (ITT) estimates, with some discussion of possible implied IV coefficients given different assumptions on what is the relevant “first stage.”
For our core analysis, we will use student-level data to run intent-to-treat OLS regressions with randomization strata and school grade fixed effects, and standard errors clustered at the school-level. For natural families of outcomes, we will sometimes create an index by standardizing each outcome and taking the simple mean. We will explore heterogeneity along four dimensions: caste, grade, gender, and school size (many schools in the sample are small, and it is not clear that a House System would have meaningful effects in schools that are small enough for everyone to already know everyone).
Using our friendship network data, we will also run dyadic regressions to explore the role of group identity (e.g. same-House, same-caste, same-religion, same-gender) on friendship formation.
In the treated schools, we also had a classroom-level sub-randomization. To be eligible for the sub-randomization, the treated classroom had to have all four Houses represented, and with at least one SC/ST and one non-SC/ST student in each House. We randomized half of these eligible classrooms to be treated. Treatment entailed randomly picking two of the Houses in that classroom, and moving SC/ST students from one House to the other, increasing the variation in SC/ST share across the Houses within that classroom.
The sub-treatment has two implications for our analysis. First, we can use the within-treatment variation to look at the effects of a student having more versus fewer SC/ST students in their House in their class (this analysis will be somewhat under-powered, though we can also combine the natural variation that already comes from random assignment to Houses). Second, when looking at the overall effects of the House System (comparing treatment versus control schools), we can explore excluding the students treated by the sub-randomization (and reweighting accordingly), since this sub-randomization creates more caste homogeneity within a House, which is not what an actual House System policy would do.