A multi-sensory tutoring program for students at-risk of reading difficulties in kindergarten and first grade: Evidence from a randomized field experiment

Läsklar (meaning “ready to read” in Swedish) is a multi-sensory literacy program for students in kindergarten and first grade, developed by the third author, Anna Aldenius Isaksson. The program is aimed at students in kindergarten or first grade who, for various reasons, are at-risk of developing reading difficulties and have trouble learning to read during regular literacy instruction. For example, students with the following difficulties have previously received the program: mental impairment and other cognitive problems, hearing impairment, language impairment, dyslexia, concentration difficulties, and emotional problems. The method has also been used in the reading instruction for newly arrived immigrant children.

The program is usually implemented by special educators or teachers, and is administered either individually (one teacher and one student) or in small groups of 2-3 students. The intended frequency and duration of the program is 3-4 times a week for about 8-10 weeks, over a total of 30-35 sessions where each session lasts about 10-15 minutes.

There are three types of sessions, which are implemented in three steps. Step 1 uses the following materials:
- 25 clay figurines. The initial letters in the names of the figurines represents 25 of the Swedish language sounds. Figurines are not available for the letters c, x, w, and z. The reason is that it could confuse the students. For example, it would be logical to think that the Swedish words “citron” (lemon) and “zebra” (zebra) would begin with s as the initial sounds are the same.
- A box with 29 compartments – one compartment for each letter.
- A separate box where the figurines are initially stored.
- A laminated sheet with the fingerspelling alphabet.

During the first session, the teacher selects three figurines and the student’s task is to place the figurine in its designated compartment, or its “house”. For the figurine to be placed into its house, students should figure out a code. The first part of the code is the sound of the first letter in the figurine’s name. For example, “ss” in “sun”. When the student figures out that the word “sun” begins with an “ss” sound, they should make the sound for a while, while at the same time fingerspelling the letter. When this is accomplished, the figurine can move into its house. Once all three figurines have been placed into their homes, the session ends. Such a session usually takes about 10 minutes. During the next session, the student must first place the previously completed figurines into their homes and then continue with three new ones. It usually takes about a month before the student will have learned most of the letter sounds, after which Step 2 starts.

In Step 2, the student practices to make the sounds of short words represented by pictures. The images are pasted onto the back of a white card. The teacher acts as a secretary to the students: when students make the sounds that the word contains, the teacher writes them on the front of the card. When this is done, the newly created word card whose front is the written word and back is a picture that represents the word, is put into a binder. Using the binder the students can practice themselves, as they are able to verify if they have read the word correctly by turning the page. The binder is filled gradually until it contains about 25 words. If the students need more practice, the old words can be tied together into a book and afterwards the students begin with 25 new words.

When the students are reasonably confident in all of the letter sounds, Step 3 uses flashcards of syllables to practice speed. Students who manage to go through also this step before the end of the 8-10 weeks go on to read the usual reading books that are used in regular reading instruction.

The theory of action behind Läsklar is that by focusing on phonemic awareness and phonics students will be helped in understanding that the audio stream coming from speaking can be divided into a number of letter sounds, and learn how to differentiate them from one another. In the beginning it will be easier for the student to remember which house the figurine should be in than to recognize the letter’s visual representation – eventually the students will be able to connect the letter with the sound of the letter. The multi-sensory training aims to use as many senses as possible in order to strengthen memory capacity. When students help to place the figurines into their homes, the kinesthetic (motion) sense is activated. The game also creates a little storyline about the figurines moving into their home, which means that the visual and episodic memories are used. The students also get to touch the figurine, meaning that the tactile sense is activated. The method furthermore allows the student to listen to the sound based on the first letter of the figurine, thereby activating the auditory sense. Several senses are also activated through the consistent use of fingerspelling, and the students can use this tool as a memory aid when they proceed to regular text reading.

Working with figurines also seems to strengthen the students’ motivation. It is easier and more fun for students to figure out where the figurines are going to live and help them into their house, than to simply think about how the beginning of a word sounds. By intervening early, hopefully before students come to see themselves as “bad readers”, the program seeks to boost students’ reading motivation and self-efficacy also in the longer run.

2016-11-01

2018-06-20

We have two primary outcomes:
1. A test of decoding skills.
2. A test of letter knowledge.

1. Decoding test. We use LäSt, which is a norm-referenced test battery used for identification and diagnosis of basic literacy skills (Elwér et al., 2016). The material includes tests of decoding skills, spelling and reading comprehension. We use a subtest of decoding skills (“Avkodning ord”), where students read individual words (not pseudo-words) for one and a half minute, and points are given for each correct word read. The test was normed on a sample of 1,043 first grade students in three regions of Sweden. The student sample was chosen to be representative of grade 1 to 6 in terms of variables such as SES, and language and foreign background (Elwér et al., 2016).

2. Letter knowledge. We use a diagnostic test from the norm-referenced test material LäsEttan (Johansson, 2009), testing how many letters of the alphabet students are able to name. The test takes about 1-2 minutes to complete. The maximum score is 27. The test was norm-referenced in 2008 using a sample of 700 students in grade 2 (Johansson, 2009).

There are four secondary outcomes: A test of phonological awareness, and three measures of motivation, enjoyment, and self-efficacy.

Phonological awareness. We use a researcher developed short test of phonological awareness that examines if students recognize the first sound in 10 words represented by a picture of an object or animal. The student is asked to first identify the animal or object depicted, and then to say the sound the word starts with. Each correct answer is worth one point, and the maximum score is therefore 10. As this test was developed by one of the researchers for this intervention (which none of the other tests were), has a low maximum score and may, especially in first grade, imply considerable ceiling effects because phonological awareness is often a strong focus of regular instruction in Swedish schools (see e.g., Bøg et al., 2017), we consider this test a secondary outcome. Because of the potential ceiling effects, we expect to find smaller effect sizes on this test.

Motivation, enjoyment, and self-efficacy. We use three simple questions posed to the participating students to measure impacts on motivation and self-efficacy (Swedish wording in parentheses):
- How easy or hard do you think it will be to learn how to read? (Hur lätt eller svårt är det att lära sig läsa, tror du?)
- How much fun do you think it will be to learn how to read? (Hur roligt tycker du att det ska bli att lära sig läsa?)
- How much would you like to learn how to read? (Hur gärna vill du lära dig läsa?)

For each question, students indicate on a Visual Analogue Scale (VAS) with the two end points represented by a sad and a happy smiley, respectively. The students are told that e.g., the sad smiley represent that it will be no fun at all to learn how to read and the happy smiley that it will be a lot of fun. The students’ marks are then transformed into a scale with starting point 0 and max point 10.

We recruited 12 schools sequentially during fall of 2016 and spring of 2017. The six first schools started to implement the program for their kindergarten students either late in 2016, or during spring 2017. The six schools recruited last began the implementation in kindergarten/first grade at the start of fall 2017. Four schools are also running a second round of the program for new cohorts. The last students will have finished their training with the program late May or early June 2018.

The procedure used to assign students to treatment and waitlist control groups had three steps: First, staff in each school, usually teachers and special educators, selected the group of students believed to be most in need of extra help, regardless of the reason for their difficulties. This group was tested using all outcome measures. Second, based on the results the school selected the final group that they wanted to include in the intervention. If this group was smaller than the full group being tested (e.g., because of resource constraints), the students with the lowest scores on our primary outcome measures were chosen. Third, using the two primary outcome measures the researchers matched students into pairs or triples, depending on the schools preferred form of implementation and on the number of students in the group. One student in each pair, or one or two students in each triple, was then randomly assigned to receive the program in a first period (i.e., was assigned to the treatment group), the rest were put on a waitlist and receive the program in a second period.

Externally hired testers (two former teachers) who are blind to treatment status test students individually three times on the short-run measures: before the program starts (baseline), after the treatment group has finished training with the program (i.e., after about 10 weeks; post-test), and after the waitlist group has also received the program (i.e., about 10 weeks later, or about 20 weeks after the first test; follow-up). As the schools were recruited sequentially, the timing of the testing is different for each school.

The matching of students into pairs/triples was done by the nearest neighbor method, where closeness is based on the two primary outcome variables. However, as no students knew how to decode words at pre-test, the letter knowledge tests determined the matches in practice. If there was a tie we used the test of phonological awareness to determine the closest match. At one school’s request (for scheduling issues), we stratified some students by class first, and then used the same matching procedure. Randomization to treatment and waitlist control groups was conducted by Jens Dietrichson in his office using the random number generator in Microsoft Excel. Students’ identities were anonymized and the only information available at randomization was the pre-test scores.

The randomization unit is always the student. For the majority of students, the treatment is given one-to-one, i.e., it is not clustered. Two schools have also given the program to some students in groups of one teacher and two students. In these cases, the treatment is clustered (although the assignment was not).

Yes

147 student groups (where 131 are single student clusters and 15 are pairs of students).

Sample size: 161 students in 12 schools.

82 students in the treatment group and 79 students in the control group.

Minimum detectable effect size: Using a power of 80%, a two-tailed p-value = 0.05, and a baseline correlation between pre-treatment and post-treatment scores of 0.7, we can detect effect sizes of 0.33. The baseline correlation between pre- and post-treatment scores is based on a pilot study using the same two primary outcome measures, which found a somewhat higher correlation than 0.7 (Bøg et al., 2017). With the baseline correlation set equal to 0, we can detect an effect size of 0.45.

Analysis plan

This study examined a literacy program that targeted students most at risk of reading difficulties in kindergarten and first grade of 12 Swedish schools. The program used multi-sensory learning methods that focused on phonological awareness and phonics, and was delivered during 10 weeks over 30–35 sessions by teachers in a one-to-one or one-to-two setting. In total, 161 students aged 6–7 years were randomly assigned to a treatment group or a waiting list control group. The treatment group showed large and statistically significant improvements compared to the control group on the two pre-registered primary outcome measures: a standardized test of decoding (Hedges’ g = 1.07) and a standardized test of letter knowledge (g = 1.03). The improvements were also significantly larger on measures of phonological awareness (g = 0.56) and self-efficacy (g = 0.57), but not on measures of enjoyment and motivation. The program appears cost-effective relative to other tutoring programs.

Martin Bøg, Jens Dietrichson & Anna A. Isaksson (2021) A multi-sensory tutoring program for students at risk of reading difficulties: Evidence from a randomized field experiment, The Journal of Educational Research, 114:3, 233-251, DOI: 10.1080/00220671.2021.1902254

https://doi.org/10.1080/00220671.2021.1902254

Working paper version of the above paper.

Martin Bøg, Jens Dietrichson & Anna Aldenius Isaksson (2019). A multi-sensory tutoring program for students at-risk of reading difficulties: Evidence from a randomized field experiment. IFAU Working Paper, 2019:7.

https://www.ifau.se/en/Research/Publications/Working-papers/2019/a-multi-sensory-tutoring-program-for-students-at-risk-of-reading-difficulties/