|
Field
Abstract
|
Before
Summary of the Evaluation Results:
We estimate whether pre-kindergarten and kindergarten children in schools randomly assigned to participate in the Big Word Club (BWC) scored higher than students in a control group of schools on an assessment of receptive vocabulary based on words included in the BWC program (the BWC Assessment). We also assessed students using a standardized test of receptive vocabulary. 818 students in 47 schools were assessed.
Results show that children in classrooms with access to the BWC for 17 weeks identified, on average, between 1.194 and 1.319 more words on the BWC Assessment compared to the control group. The intent-to-treat effect size is between .229 and .267 standard deviations depending on how scores were imputed. All estimates are statistically significant at p<0.01. In the treatment-on-treated model, treated students identified 1.638 more words on the BWC Assessment at 17-weeks than the students who were not treated. The effect size was .320 standard deviations.
We assessed students 25 weeks after the intervention began on words included in the first 17 weeks of the BWC program. At that point students in schools with access to the BWC identified between 1.071 and 1.242 more words (depending on the imputation) on the BWC Assessment than students in the control schools. This corresponds to an effect size of between .229 and .257 standard deviations with all estimates statistically significant at p<0.01. At 25 weeks the TOT estimated effect size was .305. This means that children in the treatment group retained their advantage over children in the control group for at least 8 weeks.
We also assessed students at 25 weeks after treatment using the Peabody Picture Vocabulary Test 4, a standardized test of receptive vocabulary. Children in classrooms with access to the BWC scored higher on the PPVT than students in classrooms without access, but the difference was not statistically significant at p=.05.
The estimated effect of the BWC was greater for female students compared to male students who took the 25-week BWC Assessment, but there was no difference in the effect by students’ age, whether the student was in kindergarten, whether the student had English as a second language or had special needs, or whether the school was private or received Title 1 funding. Teachers report liking the BWC, believing that it is effective, and wanting to use it in the future. The main reason that they report not using he BWC is having too many other required activities.
The effect of the BWC on receptive vocabulary is the same as or greater than the effect of the only two programs that we could find that are comparable to the BWC. The BWC may be more cost effective than other programs because it does not require teacher training or follow up.
|
After
Vocabulary is an important part of literacy skill development, and understanding the effectiveness of various types of programs can help parents, schools, and states decide what to invest in. Programs that use entertainment to engage students to learn (“Edutainment” programs) are popular, but experimental evidence on their effectiveness at teaching vocabulary is sparse. In this paper, we use an RCT to evaluate the effectiveness of a program that uses technology and entertainment to teach vocabulary to children. The program, Big Word Club (BWC), is a web-based platform consisting of animated book videos, dance videos, and music videos intended to help children learn one new word per day. Our field experiment was conducted with 818 Pre-K and Kindergarten students in 47 schools across 3 U.S. states. We randomly assigned schools to a control condition or to use the BWC platform in their curriculum. We find that the program was effective in teaching the vocabulary words that were targeted by the program (0.30 SD) after four months of use. Further, this effect persisted in a follow-up test 2 months later. Treated students also scored higher on a standardized vocabulary assessment (PPVT), but this difference was not statistically significant.
|