Behavioural Interventions to Increase Parental Engagament

Trial 1. Parents will be texted in advanced of their child's Maths test.
Trial 2. Parents will be texted a conversational prompt related to the day's Science class.
Trial 3. Parents will be texted a report of their child's attendance (academic year to date), compared to the rest of their child's year.

1. Pre-informing parents of an upcoming Maths test

Schools frequently report grades to parents via report cards. However, these grades are often reported weeks after the test, and without any reference to the date of the next test. This leaves parents with little opportunity to influence the amount of effort a child puts in when it actually matters.

In this experiment, students were assigned to either treatment or control groups. Parents of pupils assigned to the treatment group were informed of an upcoming Maths test 5 days, 3 days and 1 day in advance.

5 days and 3 days before the test date, parents were texted:

"[Parent Name], [Student Name] has a Maths test on [Test Day]. Please remind them to study and help them to prepare in any other way you can. Thanks, [School Name]."

The night before the test, parents were texted:

"[Parent Name], [Student Name] has a Maths test tomorrow. Please remind them to study and help them to prepare in any other way you can. Thanks, [School Name]."

Students in the control group sat tests as normal and parents were not texted in advance. Students in the control and treatment groups were informed of the test at the same time, and before any texts were sent to parents.

The experiment was run in 5 schools for one test in the second half of the Spring Term (February 24th to April 4th), see the Design section for more details.

2. Prompting home conversations about Science class

Parents may get discouraged from asking their child what they learned at school because previous attempts have not yielded much of a discussion (perhaps because the child is tired, has forgotten, or is not feeling particularly talkative). Alternatively, the parents themselves might not have enjoyed school and are either uninterested or not confident in discussing what was taught.

In this experiment, students were assigned to either a treatment or control group. Parents of treatment group pupils were texted a conversational prompt (written by teachers) related to the day’s Science class. The prompts followed a simple two-sentence formula – a succinct, non-technical summary of the lesson followed by a related question designed to spark curiosity in the parent.

Some examples include:

“Today [Name] revised forces and The Solar System. Ask [Name] to explain why we have seasons.”

“Today in Science [name] revised static electricity and forces. Ask [name] how a balloon gets charged when you rub it on your clothes.”

Texts were scheduled to be sent after every lesson, unless the student had more than one Science lesson per day. Parents were sent a maximum of one text per day. Control group parents did not receive any texts, schooling continued as normal.

The experiment was run in 5 schools for the second half of the Spring Term (February 24th to April 4th). More details can be found in the Design section

3. Using Social Norms to increase attendance

Communicating social norms has been found to change behavior in numerous domains. For example, people who don’t pay their taxes have been found to be more likely to pay if told that the vast majority of their neighbours have paid. Likewise, telling people that they are using far more energy than everyone else on their street has been found to reduce subsequent energy consumption. Informing someone that they are in the minority often causes them to change their behavior to conform with the majority.

Surveys indicated that parents of students with poor attendance records may be unaware of how high their absence rate is compared to classmates. These findings are consistent with biases found elsewhere; numerous psychological studies have found that the majority of people think they are in the top half of their peer group for attractiveness, intelligence and driving (among others). Combined with the social norms experiments described above, it seems plausible that informing parents of low attenders that their child’s classmates miss far fewer days of school will lead to those parents conforming to the norm and improving the attendance of their child.

In this experiment, students that have missed 4 days or more than the modal number of days in their year were assigned to one of three groups: Control, Treatment A and Treatment B. Parents of Treatment A students were texted an attendance report for the school year so far (the Winter and Spring Terms):

“Up to 4th April, [Student Name] missed [Days Missed] school days. Please help your child attend, [School Name]”

Where the number of days missed exceeded 20, the text simply said ‘more than 20’. This is because the number of days missed was estimated from the attendance record held by the school (which is recorded as a percentage) and the number of school days (which might vary from year to year).

Parents of Treatment B parents were texted an attendance report for the year so far, plus a comparison to the rest of the student’s year group:

“Up to 4th April, [Student Name] missed [Days Missed] school days - the typical Year [XX] missed [Year XX Mode]. Please help your child attend as much as others, [School Name]”

The modal number of days in that year group was used as the number of days missed by the ‘typical’ student. Again, values over 20 days were simply referred to as ‘over 20’, and when the typical student had missed 0 days, the text referred to ‘fewer than 1’ days being missed. This is because all the data for days were rounded to the nearest day, and so many modes were rounded to 0 (but were actually probably between 0 and 0.5). More details can be found in the design section.

Attendance in the first and second halves of the Summer Term will be analysed separately. We hypothesize that there will be a treatment effect in the period between the texts are sent and half-term, but that in the second half of the summer term, the treatment effect will either be greatly reduced or disappear entirely.

2014-04-24

2014-07-31

Trial 1. Maths test results (z-standardized within class) and maths test grade (z-standardized within year)
Trial 2. Science test results (z-standardized within class) and science test grade (z-standardized within year)
Trial 3. Attendance for the first and second halves of the Summer Term in the 2013/14 school year.

For trials 1 and 2, grades will be converted into within year z-scores.
For trial 3, the number of days missed will be calculated by multiplying the percentage of days missed (which is how schools conventionally store attendance data) by the number of school days in the relevant period.

Trial 1

Students are individually randomised to treatment/control groups. 5 schools make up the sample. Block randomisation ensuring balance across treatment control groups for: school, year group, class, prior attainment, free school meal status. Parents are texted 5 days, 3 days, 1 day in advance of a maths test. Students are surveyed for feedback after the test. Test grades are then compared for differences in within year z scores.

Trial 2

Students are individually randomised to treatment/control groups. 5 schools make up the sample. Block randomisation ensuring balance across treatment control groups for: school, year group, class, prior attainment, free school meal status. For 4 to 6 weeks, parents are conversational prompts related to the day's science class. Students are surveyed for feedback after their end of term test. Test grades are then compared for differences in within year z scores.

Trial 3

Students are individually randomised to treatment A/treatment B/control groups. 10 schools make up the sample. Block randomisation ensuring balance across treatment control groups for: school, year group, prior attendance. Treatment A parents are texted number of days missed by student in previous two terms. Treatment B texted Treatment A + comparison to typical (modal) student in that school/year. Attendance across two half-term blocks of the Summer term compared at the end of the academic year.

Trial 1

Outcome Variables

Maths test result (within class z-score), Maths test Grade/Attainment level (within year z-score), attitudes towards parent engagement and maths in general (survey responses). Note that 3 different surveys were conducted, (see appendix B).

Outcome Variables: Issues

- All schools do not keep a record of test results once they have been standardized into attainment levels (which are not as granular as test scores). Therefore there is risk that small effects will be missed if looking at attainment levels only.
- Survey data is stated rather than revealed preference.

Sample

5 secondary schools, year groups 7 to 11 inclusive. Students were removed from the study if their parents refused permission to participate or if the school had no working mobile phone number for any parent.

Randomisation

Randomisation was done at the student level to maximize power and because although some contamination was likely, the intervention only ran for 5 days and hence was unlikely to completely undermine the randomization. Block randomization was used, so students were sorted by class, free school meal status and prior attainment, then randomized within those blocks. This was done to ensure the treatment and control groups would be balanced on observables. The Stata randomization algorithm has been kept by RC.

Data

Outcomes data provided by school. Covariates data gathered from the school and found in the National Pupil Database (NPD). Survey data administered and entered by RAs.

Trial 2

Outcome Variables

Science test results (within class z-score), attainment level (within year z-score), attitudes towards parent engagement and science in general (surveys). Note that 2 different surveys were conducted, see appendices.

Outcome Variables: Issues

Not clear that all schools keep a record of test results once they have been standardized into attainment levels, which are not as granular as test scores. Risk that small effects will be missed if looking at attainment levels only.

Sample

5 secondary schools, year groups 7 to 11 inclusive. BTEC students were dropped from the study, as were Separate Science students in one school because scheduling the texts according to the timetable became too complicated due to extent to which students sat in different classes on different days. Students were removed from the study if their parents refused permission to participate or if the school had no working mobile phone number for any parent.

Randomisation

Randomisation was done at the student level to maximize power and because although some contamination was likely, the intervention only ran for 6 weeks and hence was unlikely to completely undermine the randomization. Clearly, the risk of contamination was higher than in Experiment 1. Block randomization was used, so students were sorted by class, free school meal status and prior attainment, then randomized within those blocks. This was done to ensure the treatment and control groups would be balanced on observables. The Stata randomization algorithm has been kept by RC.

Data

Outcomes data provided by school. Covariates data gathered from school and National Pupil Database (NPD). Survey data administered and entered by RAs.

Trial 3

Outcome Variables

Attendance rate in first and second half of summer term.

Outcome Variables: Issues

Sample

10 secondary schools, year groups 7 to 10 inclusive. Students were only included if they had missed 4 days or more than the modal number of days missed in their year group. Students were removed from the study if their parents refused permission to participate or if the school had no working mobile phone number for any parent. Students were also withdrawn if schools felt they had been absent because of extenuating circumstances (such as serious health problems or family distress). ‘Extenuating circumstances’ was left to the school’s discretion.

Randomisation

Randomisation was done at the student level to maximize power and because although some contamination was likely, the intervention was composed of only 1 text message and hence was unlikely to completely undermine the randomization. Block randomization was used, so students were sorted by year group and prior attendance records, then randomized within those blocks. This was done to ensure the treatment and control groups would be balanced on observables. The Stata randomization algorithm has been kept by RC.

Data

Outcomes data provided by school. Covariates data gathered from school and National Pupil Database (NPD). Survey data administered and entered by RAs.

Randomisation done using Stata. Raj Chande holds randomisation code used. Individual randomisation, blocking by covariates.

Student level.

No

Trial 1: 5 schools
Trial 2: 5 schools
Trial 3: 10 shools

Trial 1: 5000. Trial 2: 5000 Trial 3: 5000

Trial 1: 50/50 split
Trial 2: 50/50 split
Trial 3: 33/33/33 split