Impact Evaulation of Jeg Vet Program

We test whether the learning tool "Jeg Vet" increases children's theoretical knowledge about violence and abuse for primary school children in grade 2-7. Learning Tool “Jeg Vet” is a free, digital learning tool to help children in kindergartens and schools to learn about bullying, violence, abuse, and children's rights. The content of "Jeg Vet" is aligned with the framework plan for kindergartens and the curriculum for schools. "Jeg Vet" was developed by Salaby, a digital learning universe for kindergartens and primary schools, in collaboration with Save the Children, commissioned by the Norwegian Directorate for Children, Youth, and Family Affairs.

Classes in each grade of the recruited schools are randomly assigned to either the treatment group (using "Jeg Vet") or the control group (not using "Jeg Vet"). Children in both treatment and control groups complete surveys at three different time points over one school year to assess the development of their knowledge about violence and abuse.

The classes in the treatment group will use the "Jeg Vet" learning tool for approximately two school hours in October 2024 and March 2025.
The following modules have been selected in the intervention:
Grades 1-4:
1) "The Body is Yours"
2) "Right to be Safe"

Grades 5-7:
1) "What is Child Welfare Services?"
2) "Safe at Home"

2024-10-08

2025-04-01

1. Students, grades 2-7: Overall program-related knowledge.

1. Students, grades 2-7: Overall program-related knowledge. Composite indicator including understanding of phenomenon and theoretical action competency on domestic violence and sexual abuse, as well as knowledge about Child Welfare Services. Measured as percentage of correctly answered questions across measurement using self-developed knowledge scales inspired by Bustamante et al. 2019, Dake et al. 2003, CKAQ-short (Tutty 2020), “What if” situations test (Wurtele et al. 1998), Berger 2017.

Student outcomes
2. Students, grades 2-7: Knowledge about domestic violence
3. Students, grade 2-4: Knowledge about sexual abuse
4. Students, grade 5-7: Knowledge about Child Welfare Services
5. Students, grades 2-7: Understanding of phenomenon violence
6. Students, grades 2-4: Understanding of phenomenon sexual abuse
7. Students, grades 2-7: Theoretical action competency violence
8. Students, grades 2-4: Theoretical action competency sexual abuse
9. Students, grade 2-7: Fear of Child Welfare Services
10. Students, grade 5-7: Specific self-efficacy
11. Students, grades 2-7: Differences in effects across grade level, gender, minority background and socio-cultural capital

Teacher outcomes
12. Teachers: Willingness and confidence in ability to teach children about violence and sexual abuse
13. Teachers: Strength of concern about students’ well-being in the present class during past 2 weeks
14. Teachers: Number of cases of concern about students’ well-being in the present class during past 2 weeks
15. Teachers: Having reported concern about students in the present class to superiors during past 2 weeks
16. Teachers: Assessment of students’ behaviors during intervention and usefulness of learning tool

Student outcomes
2. Students, grade 2-7: Knowledge about domestic violence. Percentage of correctly answered questions across measurement using self-developed knowledge scale inspired by Bustamante et al. 2019, CKAQ-short (Tutty 2020), “What if” situations test (Wurtele et al. 1998).
3. Students, grade 2-4: Knowledge about sexual abuse. Percentage of correctly answered questions across measurement using self-developed knowledge scale inspired by Bustamante et al. 2019, CKAQ-short (Tutty 2020), “What if” situations test (Wurtele et al. 1998).
4. Students, grade 5-7: Knowledge about Child Welfare Services. Percentage of correctly answered questions across measurement using self-developed knowledge scale inspired by Berger 2017.
5. Students, grades 2-7: Understanding of phenomenon violence. Percentage of correctly answered violence questions pertaining to understanding of phenomenon.
6. Students, grades 2-4: Understanding of phenomenon sexual abuse. Percentage of correctly answered sexual abuse questions pertaining to understanding of phenomenon.
7. Students, grades 2-7: Theoretical action competency on violence. Percentage of correctly answered questions across violence questions pertaining to theoretical action competency.
8. Students, grades 2-4: Theoretical action competency on sexual abuse. Percentage of correctly answered questions across sexual abuse questions pertaining to theoretical action competency.
9. Students, grade 2-7: Fear of Child Welfare Services. Percentage of “Yes” answers to self-developed survey question inspired by Berger 2017.
10. Students, grade 5-7: Specific self-efficacy. Measurement using self-developed survey questions inspired by Dake et al. 2003. Mean score of agreement for each item on 4-point Likert scale.
11. Students, grades 2-7: Differences in effects across grade level, gender, minority background and socio-cultural capital. Measured by assessing effects separately for subgroups identified through registered grade level and self-reported gender, use of non-Norwegian language at home and number of books at home.

Teacher outcomes
12. Teachers: Willingness and confidence in ability to teach children about violence and sexual abuse. Measured using provider attitudes and self-efficacy scale inspired by Guastaferro et al. 2021. Mean score for each item on 5-point Likert scale.
13. Teachers: Strength of concern about students’ well-being during past 2 weeks. Measured using student concern scale inspired by the Swedish SOFIA study (see Svensson et al. 2015). Mean score for each item using 5-point Likert scale.
14. Teachers: Number of cases of concern about students’ well-being in class during past 2 weeks. Measured using self-designed survey question posed to respondents who score more than 0 on the concern scale. Average number of cases reported by each teacher.
15. Teachers: Having reported concern about students in the present class to superiors during past 2 weeks. Measured using self-designed survey question posed to respondents who score more than 0 on the concern scale. Percent “Yes” responses.
16. Teachers: Assessment of students’ behaviors during intervention and usefulness of learning tool. Measured using self-developed survey questions. Mean score for each item on 5-point Likert scale.


References
Berger, T. F. (2017). Barns oppfattelse av det norske barnevernet. En spørreundersøkelse blant norske 5.-og 9. klasseelever (Master's thesis, UiT Norges arktiske universitet).
Bustamante, G., Andrade, M. S., Mikesell, C., Cullen, C., Endara, P., Burneo, V., Yépez, P., Avila Saavedra, S., Ponce, P., & Grunauer, M. (2019). “I have the right to feel safe”: Evaluation of a school-based child sexual abuse prevention program in Ecuador. Child Abuse & Neglect, 91, 31–40. https://doi.org/10.1016/j.chiabu.2019.02.009
Dake, J. A., Price, J. H., & Murnan, J. (2003). Evaluation of a Child Abuse Prevention Curriculum for Third-Grade Students: Assessment of Knowledge and Efficacy Expectations. The Journal of School Health, 73(2), 76–82. https://doi.org/10.1111/j.1746-1561.2003.tb03576.x
Guastaferro, K., Font, S. A., Miyamoto, S., Zadzora, K. M., Walters, K. E., O’Hara, K., Kemner, A., & Noll, J. G. (2023). Provider Attitudes and Self-Efficacy When Delivering a Child Sexual Abuse Prevention Module: An Exploratory Study. Health Education & Behavior, 50(2), 172–180. https://doi.org/10.1177/1090198121997731
Svensson, B., Andershed, H., & Janson, S. (2015). A Survey of Swedish Teachers’ Concerns for Preschool Children at Risk of Maltreatment. Early Childhood Education Journal, 43(6), 495–503. https://doi.org/10.1007/s10643-014-0684-z
Tutty LM. Children's Knowledge of Abuse Questionnaires (CKAQ)-Short: Two Brief Ten-Item Measures of Knowledge about Child Sexual Abuse Concepts. J Child Sex Abus. 2020 Jul;29(5):513-530. doi: 10.1080/10538712.2019.1688443. Epub 2019 Nov 21. PMID: 31751187.
Wurtele, S. K., Hughes, J., & Owens, J. S. (1998). An Examination of the Reliability of the "What If" Situations Test: A Brief Report. Journal of Child Sexual Abuse, 7(1), 41–52. https://doi.org/10.1300/J070v07n01_03

About 23 Norwegian schools have been recruited for the study, and all children in grades 2-7 in each school will participate.

For the treatment classes, one “Jeg Vet” learning module will be implemented in October 2024 and the other in March 2025. The order of the modules to be implemented is randomly assigned by the project team, so half of the schools will run the first module and the other half will run the second module in October 2024. Within the same school, all classes follow the same order of modules.

All classes in grades 2-7 in the recruited schools, both treatment and control classes, will participate in surveys at three different time points during the 2024-2025 school year: in September 2024 (before the first intervention), November 2024 (after the first intervention), and April 2025 (after the second intervention). Three surveys will be conducted by research assistants recruited and trained by the project team at Fafo.

The survey will evaluate children's knowledge about violence and abuse before and after the intervention. Control group children will also complete the surveys at the same time as the treatment group children to evaluate the development of their knowledge over time without the intervention.

Randomization of assigning treatment and control groups is conducted at the grade level within each school. If there is an odd number of classes in a grade, treatment groups are oversampled. That means that if there are 2, 4, or 6 classes in a grade, half of the classes will be randomly assigned to the treatment group and half to the control group. If there are 1, 3, or 5 classes in a grade, more classes are assigned to the treatment group than to the control group.

Randomization of schools for the order of "Jeg Vet" modules is done on excel file with rand() function
Randomization of classes within each grade for "treatment" and "control groups" is done in SPSS with RV.univform () function

Unit (clusters) of randomization: classes in each grade

Yes

23 schools to be recruited
All the classes in grade 2-7 in these schools
Number of classes varies between 10-30
Rough estimated number of class:350-400 classes

The class size varies between 10-30 The estimated number of student: 6,000 students

Grades 2-4 (estimated):
1700 students (110 classes) in treatment group; 1300 students (80 classes) in control group

Grades 5-7 (estimated):
1700 students (110 classes) in treatment group; 1300 students (80 classes) in control group

Analysis Plan