Envision RCT

The partner for this trial is Envision. It is a charity that was set up in 2000 to challenge the stereotype of ‘the youth’ as apathetic and disengaged with the issues going on around them. Envision’s 10 month social action programmes does this by supporting young people to design their own local community projects tackling issues ranging from street crime to climate change. Today Envision’s programme supports 16-19 year olds from over 130 schools and colleges in Birmingham, Bristol and London.

2014-01-12

2014-08-31

We have multiple outcome measures for this trial, reflecting both the multiple objectives of this programme, and the need to rely on proxies for some outcomes. Our battery of outcome measures comprises both hard (objective, realised behavioural) and softer (self-reported or more subjective) outcomes.
The majority of our hard outcome measures are educational (i.e. attainment, attendance, behaviour) and will be sourced from the National Pupil Database (NPD). The National Pupil Database (NPD) is a pupil level database which matches pupil characteristic data, such as gender, ethnicity and eligibility for free school meals, and school characteristics data to pupil level attainment.
Since the number of participants per school will be relatively small (meaning that using school level identifiers would be subject to considerable noise), we will track individual outcome measures using the unique pupil numbers (UPNs) of individual participants. All pupils have a UPN that can be linked to their performance through the NPD. UPNs stay with participants if they leave the school, and so we will be able to track them even if this happens, reducing dramatically the risk of attrition . Schools will provide the researchers with students’ UPN through their School Information Management System (SIMS) at the beginning of the trial. These numbers will be used to monitor outcome measures and attrition at the individual level, while maintaining the anonymity of the participants.
The NPD will also aid us in our data analysis as it will provide a rich set of measures that capture individual and school level characteristics, which will be useful when performing balance checks or sub-group analysis, as well as allowing us to control for time invariant school level effects (such as teacher quality or school culture). .
Our soft outcome measures will be captured by the survey based Quality Outcomes Framework commissioned by the independent Campaign for Youth Social Action (renamed Step Up To Serve following the November 2013 launch). This framework (please see Table X for an outline) is comprised of a set of metrics designed to assess perceived employability by capturing measures for a number of headline skills such as self-control , grit and resilience . We will be assessing some of these outcomes by administering a pre and post intervention survey to all participants in the control and treatment groups. A detailed description of the criteria used to select the outcomes that are included in the surveys can be found in APPENDIX […]
We will also be including in the survey 4 questions for well-being developed by ONS as another soft outcome of the programme. This is an important outcome in itself and provides at least one outcome measure that is consistent with outcomes measured across other youth programmes.

This trial will use a pipeline design . In this trial design the treatment of some groups is lagged to create a control condition, with the first group allocated to receive the intervention acting as the treatment group and all the groups allocated to receive the intervention later acting as the control. This design may be thought of as the simplest formulation of a stepped-wedge design .
We have chosen this design as a way to incentivise schools in the control condition to participate in the evaluation, as they still receive the intervention only at a later date. In addition, a two year pipeline design with random assignment between year 1 and year 2 maximises the statistical power available for the trial, as well as providing an incentive for groups assigned to receive the intervention in year 2 of the trial to adhere to the trial during the first year. To support the need to recruit additional schools, ENV will pay a financial grant (to be paid at the end of the trials so as not to interfere with the evaluation) to all schools that are allocated to year 2.
The participants in the trial will be recruited from secondary schools across. The programme will be implemented in 10 schools in year 1 (treatment group) and in 10 schools in year 2 (control group). Our comparison of interest is therefore in the difference in outcomes between our control and treatment groups at the end of year 1.

Randomisation will be conducted in an office by a computer. Randomisation will be conducted by Michael Sanders (Behavioural Insights Team & KSG)

Randomisation will be conducted at the cluster level, where the clusters are individual schools signed up to the trial.

Yes

20 schools

100 observations (pupils) per school

10 Schools control, 10 schools treatment