Empowering Staff to Enhance oral language in the early years

Intervention
Talking Time is an oral language intervention with, as a primary outcome, the language and communication development of preschool children. In this study it is delivered by appropriately trained educational professionals who have received specific professional development (PD) from the research team. There is also strong evidence to suggest that oral language skills impact on social, emotional development (SED) and so we examine this possibility by including a measure of SED as a secondary outcome.

The Talking Time arm of the study will be compared with a "Business as usual" arm. It is predicted that there will be no differential impacts on the non-targeted skills (non-verbal ability and phonology).

The intervention will run from January to May 2022.

Intervention Content: The manualised component of the intervention will be based on Talking Time. Three main activities will be included in the trial:
1. Word Play involves a series of dramatic activities and games using target vocabulary.
2. Story Conversations supports children in talking about the pictures in a book to draw inferences using language and to develop complex vocabulary and grammar.
3. The Hexagon Game provides children with a visual stimulus to support the construction of personal oral narratives.
Acting out and Story talk will be implemented over the full intervention period. The Hexagon game, which emphasises narrative language, will be added to the range of activities in week 8 of the intervention period (as per original protocol).

The PD element of the trial will include three components:
1. Materials and training to prepare practitioners to use the intervention activities and materials, and make the language-learning principles behind them explicit. TT already includes training guidance and videos of staff carrying out the activities and this will serve as the starting point, supplemented by additional DVD footage to be developed and piloted.
2. Support sessions from mentors to provide feedback on TT practice, with a focus on recognition and acknowledgement of specific language supporting behaviours, with the aim of increasing both teachers’ understanding of the principles behind these behaviours and the likelihood that they will apply them within their practice with other activities.
3. In addition to mentor-supported analysis of practice, teachers will also be encouraged to discuss and share practice with each other, to build a strong "community of practice" using the TT intervention.

Delivery: In the autumn term (2021) all classes will be provided with a whole school CPD session on the impact of early oral language delivered by the PIs. This session will also outline the assessment measures that will be used in the classes. The intervention group will receive a further three delivery sessions (one covering each of the three activities and associated principles) and will be provided with exemplar materials to support the activities. Training will be provided by independent appropriately qualified trainers.

Posttest data will be collected in the summer term (2022) on completion of intervention. This provides the necessary time gap to conform with test restrictions for the standardised measures.

End of intervention:
To acknowledge their involvement all participating schools will be given story books following successful collection of the follow-up data. Following the post-test data collection BAU classes will be provided with the intervention materials and the option of a training session by both the PI or CoIs. A final interactive conference will be held for all participating schools in the north and the south.

General Approach:
A broad set of language skills, including vocabulary, narrative and language comprehension, need to be targeted to support language skills in vulnerable populations. Talking Time was designed to explicitly combine three core oral language skills, detailed below, in a manualised approach to support implementation in nursery classes. The activities were all designed to be fun, engaging and developmentally appropriate. The manual contains details of activities, suggestions for developing context specific games, worksheets to develop activities, examples of ways to talk with children to extend their language and a DVD of these activities in practice in nursery classes.
Vocabulary Support: Meta-analyses have consistently demonstrated large effects for acquisition of targeted vocabulary, although effects on standardised vocabulary tests are smaller but also significant. Moderator analyses indicate that there are greater effects when adults are trained in providing vocabulary support that combines pedagogical strategies including explicit and implicit instruction (Marulis & Neuman, 2010). TT provides staff with specific game like activities to reinforce and extend children’s vocabulary. Core vocabulary items are identified in discussion with staff.
Narrative Children’s vocabularies support their ability to create oral narratives, an important means of communication reflecting the academic language used in schools. The ability to recount an activity or describe a situation provides children with the basis for negotiating social situations and developing written language. Personally themed stories that are age appropriate can strengthen children’s use of expressive language and story grammar (Petersen & Spencer, 2016). There is no current meta-analysis of the impacts of narrative interventions but individual studies typically report small effect sizes. Narrative activities are explicitly embedded in TT through a hexagon game where children can recount and extend descriptions of familiar events. TT provides staff with exemplars and the ability to develop their own events to reflect the children’s personal contexts.
Comprehension: Meta-analyses have demonstrated that interactive (dialogic) book reading is effective in improving both expressive and receptive oral language with moderate to large effect sizes (Mol, Bus & de Jong, 2009). Moreover shared book reading generates more grammatically rich constructions than toy play. The impact of book reading is influenced by book type where books with no words generate more oral discussion and expository texts generate greater grammatical variety. TT includes interactive book reading to support oral language comprehension with books identified to extend vocabulary and the use of grammatically rich constructions. These are introduced in the training.
The three activities were produced together as an oral language intervention called ‘Talking Time', designed for all children in small groups twice a week for a 15 minute period. Tasks were designed to foster communicative exchanges with the children using activities appropriate to the children’s developmental needs, cultural contexts and to staff constraints. The aim was to use talk that focused on what the child was doing, was relevant and related to what the child was saying and that acknowledged the child’s utterances.
To support the teachers’ language use, specific emphasis was placed on the use of evidence based ways of effective talking with children. These were:-
(1) contrasts that highlighted differences in lexical items and in syntactic structures,
(2) open questions and expanding or recasting the children’s utterances,
(3) modelling language structures that the children were not yet producing and
(4) event casting where the adult provided a description of the activity to take place.

Since the initial trial of TT a modified version was used as the early years intervention for the EEF Talk of the Town project, demonstrating the feasibility of using the approach in a large-scale trial. Although the impact of the intervention on oracy skills in the early years was not assessed as part of the main evaluation it was included in the external narrative review. Changes in language, language learning environment and staff practices were reported (Menzies, 2016) supporting a wider trial. In addition, the review identified a number of features that are relevant to such a wider trial. Staff noted the need for flexible, practical support to enhance their ability to implement the activities. As such, we have enhanced the trial with a PD development component.

The proposal in this study is to embed PD within the TT materials in three ways. Firstly to act as a vehicle for developing knowledge and skills, given the evidence that practitioners need a solid foundation of knowledge about oral language to be effective in helping children develop these skills (Schachter, 2015) and on the basis that high quality content is an important characteristic of effective PD (Desimone, 2009). Secondly, to include feedback opportunities to staff, through live observations and video feedback. Video feedback has been shown to be an effective method of improving interactions skills resulting in more sensitive and verbally stimulating behaviour from teachers (Fukkink & Tavecchio, 2010). Analysing classroom videos may help practitioners apply positive practices to their classroom. Thirdly, practitioners will be encouraged to form professional learning networks to further analyse and reflect on their practice, building on theories of professional learning networks and communities of practice.

a) Language skills (operationalised as vocabulary, oral language comprehension and expressive narrative) (primary outcome)
b) Social, emotional and behavioural competencies of children in the intervention group (secondary outcome).
c) There will be no differential impacts on the non-targeted skills (non-verbal ability and phonology)

2022-01-01

2022-05-27

Child level: Oral language (vocabulary, narrative and receptive language) will be measured by a battery of standardised assessments that are commonly used in the literature and administered to children individually. These measures have been standardised on a representative UK population and the BAS3 provides Rasch ability scores to ensure that progress can be benchmarked in relation to item difficulty. Using the BAS3 for both verbal and nonverbal outcomes has the added benefit of consistency of standardisation across key targeted and non-targeted skills.
The battery consists of: Naming Vocabulary and Verbal Comprehension subtests from the British Ability Scales: 3rd edition (BAS3; Elliot & Smith, 2012); a bespoke list of vocabulary words taken from the Object and Action Naming Battery (Druks & Masterson, 2000), sentence repetition from the Grammar and Phonology Screening test (GAPS; Gardner, Froud, McClelland & van der Lely, 2006) and the Quick Interactive Language Screener (QUILS; Golinkoff, de Villiers, Hirsh-Pasek, Iglesias, Wilson, Morini & Brezack, 2017). The Bus Story test (Renfrew, 1995) is administered at follow-up only.
Additionally, parent report of child language will be measured by a shortened version of the Language Use Inventory (LUI; O’Neill, 2009).
Setting level: Quality of adult-child interactions in the classroom will be measured by audio recording of small group book sharing and toy playing sessions. Staff knowledge and attitudes will be measured by a bespoke questionnaire.


SECONDARY OUTCOMES
Setting measures: Following previous work (Mathers et al., in progress), a subset of relevant scales from the ECERS-3, ECERS-E and SSTEW rating scales will be complemented by the communication supporting classrooms tool (Dockrell et al., 2015) which examine language-learning opportunities and interactions.

Social emotional behavioural development: The Strengths and Difficulties Questionnaire (Goodman, 1997) is our proposed measure given its strong psychometric properties and availability in the range of languages. Parents and teachers will be asked to complete the questionnaire.

Children to establish whether the target children's language skills change to those of children in the comparison group receiving standard schooling or "business as usual".

None of the outcomes will be constructed de novo but the LUI identified above will be simplified somewhat to make it more relevant and will include an additional column to tap into performance in the home language in addition to English.

Parent reported social behaviour will be measured by the Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997).

The secondary measure relates to the socio-emotional development of the children.

The overall aim of the study is to ascertain whether, in areas of disadvantage, training staff to use a manualised intervention in combination with PD about language development improves children’s oral language skills in comparison with business-as-usual (BAU); and whether it can impact on overall quality of the language-learning environment beyond the intervention sessions themselves.

The project started in July 2021, nurseries were identified in June/July 2021. Parental consent was gained in September/October 2021. Data collection will run from October 2021 to December 2021. Initial training sessions will start in December 2021. The intervention and mentoring will run in the Spring term of 2022 (January to May). The children will be retested in the summer term of 2022. The data analysis, report writing and dissemination will take place between July and December 2022.

The study design aims to conform to CONSORT guidelines for reporting of RCTs. A two arm trial with randomisation at the level of the setting (cluster randomised) will provide a generalizable account of the efficacy of the intervention for all children in the final year of nursery (ages 3 to 4) in at risk populations in comparison to standard teaching practices (BAU). We will work in two areas in the country (Greater London and Teeside) with schools who postcode falls into the most deprived quintile based on the IDACI. The locations have been identified to test applicability and aid generalisability, which will enhance the robustness of the findings. All baseline measures (children, staff and setting) will be collected prior to randomisation and again following the completion of the training. There will be full reporting of selection, allocation and attrition of participants and settings.

Identification of classes and sampling: Within the two target areas, nursery schools and primary schools with nursery classes with the highest numbers of children at risk, as defined by the Income Deprivation Affecting Children Index (IDACI), will be identified; over two-thirds of disadvantaged 3 and 4 year-olds are in the maintained sector. Schools will be approached to participate, and the rationale and study design outlined. For each setting, we will collect setting details (including proportion of children eligible for pupil premium, previous OFSTED reports), local area initiatives, provision of school based speech and language therapy and child data (including EAL and SEN status).
Background information about individual staff will be collected using validated questionnaires including factors that may act as moderators or mediators of change (e.g. years of experience, training self-efficacy, openness to change, knowledge about language-supporting strategies) to explore their impact, (Ottley et al., 2015). Schools will be asked to distribute project packs to parents of all the children in the target age range (i.e. three to four years).

All data from quantitative measures will be initially screened for missing values, univariate outliers, skewness and kurtosis. Missing values analyses will be conducted to determine departures from randomness. Conventional linear transformations will be used to normalize distributions. Where outliers remain, we shall consider modelling outcomes with different link functions or replacing their values by less extreme scores.

Each pupil measure administered at initial and final assessment is likely to vary to some extent with the chronological age of the child and with time in the school year when the child was assessed. For this reason factor variables will be standardized residuals obtained by regressing each separately against chronological age and time in school year.

Promoting retention
Maximising retention will involve ensuring that we continue to communicate with all schools throughout the trial period to ensure they do not lose interest or contact with us. We will provide a named contact to all schools should they have any queries at any point throughout the evaluation. We will minimise the burden on schools, especially those allocated to the BAU group. We will clearly communicate intervention schedules and data collection schedules, while at the same time being as flexible as possible to accommodate the schools’ needs. Testing at the end of the trial period will be necessary, but we will minimise the work that this involves for schools. Outcome data will still be collected from participants who discontinue or deviate from the intervention using intention-to-treat (ITT) analysis.

Data Management
Child level data will be anonymised with a random ID number given to participants and parent questionnaires. Participant name and ID will be kept separately in a locked cabinet, in order to allow for data linkage at the post-test. Teachers will also have their own unique identifier for completing the online questionnaire (hosted on REDCap) which will allow the data to be linked pre- and post-test. All researchers on the project, including temporary RAs will complete Information Governance Assurance training. RAs will first complete scoring of child level assessments by hand and then enter it into excel spreadsheet at each site (with only the random ID number as identifier). Child level data will be entered weekly, with data validation checks in place (e.g. range checks, look-up table for acceptable data, presence check). Data from the Teesside sites will be sent to UCL in encrypted emails and merged with data from the Greater London sites. The merged data will be stored securely on the UCL Data Safe Haven, which complies with the ISO27001 information security standard and NHS Digital's Data Security and Protection Toolkit. An initial verification check of the baseline data will be completed by December 2021 by one of the research team.

Analytic plan
1. Schools will not be considered fully recruited for the purposes of the impact evaluation until all required pre-trial data have been collected from schools, consent procedures have been carried out, and pre-testing has been carried out.
2. The primary outcome will be specified as an intention to treat (ITT) analysis of the baseline language measures. We will also explore the impact on compliers using a complier average causal effect (CACE) analysis (Angrist et al., 1996) based on an agreed definition of compliance to be included in the trial protocol.
3. Other available school- and pupil-level characteristics between the treatment and control groups; pupil-level covariates will also be used to improve precision in the analysis model, while, given the clustered nature of the trial, we will also include school-level fixed effects in the model and calculate cluster-robust standard errors at the school level.
4.The analysis model will be fully specified in the statistical analysis plan and changes will not be made once outcomes data have been collected.
5. Planned pupil-level data would include, for example, non-verbal ability, gender, whether English is an Additional Language. This data will be used for exploratory sub-group analyses to explore whether there is evidence of child (language level, SEN status, gender, EAL) or class factors (such as staff qualifications), which limit response to the intervention.
6. Fidelity checks for the intervention will occur in the spring term 2022 (i.e. during the intervention phase of the study) for each of the activities.
7.Missing data will be investigated to determine any non- random nature.

Confidentiality
All data will be anonymised and the key kept in a locked storage base at the university separate to all data files. A list of children scheduled for assessment will be left at the school reception at the end of each assessment session, therefore no identifiable information will leave the premises. A data safe haven will be used to store the electronic raw data and conduct analyses securely. Data will be merged from the two sites using encrypted digital files within password protected folders. Only the necessary individuals will have access to the demographic data containing names and dates of birth (JD, JL, SM, JC, CF, KD). No names of children, teachers or schools will be published in the research findings.

Reporting
Reports to the funder will be provided and all researchers acknowledged in the report. Summary reports will be sent to all schools in the summer term following the intervention. Publications in peer review journals will occur with the lead author taking the major responsibility for creating the first draft, including all PIs as authors. A conference for all participating schools will be held in September 2022, in conjunction with the UCL Centre for Inclusive Education with a parallel meeting in the north east. This will provide an opportunity to share research findings with the schools and for schools to share their experiences of the intervention, helping to shape best practice in future studies.

This study is a school-level cluster superiority randomised controlled trial. The randomisation process will follow ‘CONSORT guidelines for reporting of RCTs’ include cluster extension (see http://www.consortstatement.org/extensions?ContentWidgetId=554).

Nursery classes will be randomly allocated to the following conditions in approximately equal proportions (by location but not by setting because they vary in size):
• Intervention: Receipt of Talking Time training in Spring 2020
• Waitlist: "Business as usual" until after outcome data collection in Summer 2020

Consenting nurseries (clusters) will be randomly numbered and then those numbers (nurseries) will be randomly drawn and the nurseries then randomised to the intervention or control groups using the minimisation* technique, stratifying by; geographic area (North and South), and IDACI with a median split. These data will be collected at the initial assessment of the children prior to randomisation.

The randomisation will be carried out bunder the guidance of but not by the project statistican (Robert Rush) so that he remains truly independent of the allcoation when it comes to the analysis. The randomisation will be carried out based on data provided by the research team.

Allocation will be single blind with concealment at the assessment level. Research Assistants collecting the outcome measures at baseline and follow-up will be blinded to the allocation of nurseries. Those carrying on the training of the nursery staff will be aware of the randomsation status of the nurseries concerned.

We will carry out randomisation within stratification blocks to reduce the risk of imbalance on important characteristics between our resulting treatment and control groups. These stratification blocks will be formed by balancing approximately equally-sized groups in our two geographical areas.

We will carry out randomisation within stratification blocks to reduce the risk of imbalance on important characteristics between our resulting treatment and control groups. These stratification blocks will be formed by balancing equally-sized North and south groups.

This is a cluster randomised study with the "setting" (nursery) as the unit of randomisation.

Yes

Sample Size: Sample sizes are estimated from current DfE data about average setting size and previous research. Maintained classes have an average of 28 pupils in a nursery class. As this is a universal intervention that is part of routine practice previous work suggests that between 60 and 80% of the parents will opt in to the collection of child level data and continue for the duration of the project. Our target number of pupils per setting is 20 for collecting child level data. Language outcome data is not parent dependent but we anticipate that behaviour data will be less, as it depends on parents/teachers completing questionnaires (see section on quality assurance). Given previous similar studies offering universal interventions, we anticipate in addition an attrition rate of 10% of classes (see Bleses et al, 2017).

640 participants with 320 in each of the two locations, 36 classes, split equally between the two sites (London and the Teesside) with each child being assessed before (October to December 2021) and after (April to July 2022) the study.

Two arms, 36 clusters.

The initial trial of TT achieved effect sizes (Cohen’s d) between .15 to .68 for oral language outcomes, with 90% participation in the intervention setting. A power calculation using G*power to detect an effect size D>.20 including baseline measures of structural language, pragmatic language and behaviour with an alpha set at p<.05 suggests that the trial requires N = 600 participants. Sampling 20 three year olds per setting and anticipating a further 10% loss off classes over the period of the project our intended recruitment is 36 classes (of which 18 are the BAU comparison).