Purpose, Practice and Teacher Transformation - An RCT of a “Learning to Learn” Teacher Training in Ugandan Secondary Schools

Last registered on November 01, 2023

Pre-Trial

Trial Information

General Information

Title
Purpose, Practice and Teacher Transformation - An RCT of a “Learning to Learn” Teacher Training in Ugandan Secondary Schools
RCT ID
AEARCTR-0012347
Initial registration date
October 24, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
November 01, 2023, 2:43 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
University of Chicago

Other Primary Investigator(s)

PI Affiliation
University of Chicago
PI Affiliation
University of California, San Diego
PI Affiliation
University of Chicago

Additional Trial Information

Status
On going
Start date
2021-06-01
End date
2026-12-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Nourani et al (2020) find large impacts of a novel general skills teacher training in Ugandan primary schools. The teacher training program is now being implemented in secondary schools and coincides with a comprehensive change in towards a competency-based approach to education in the national curriculum for secondary education. Using this context, this project advances two broad research questions: 1) does effective implementation of pedagogies associated with the competency-based curriculum require teachers to understand their purpose? 2) do the effects of the general skills approach to primary education replicate at secondary? This pre-analysis plan proposes a three-arm design for a follow-up randomized controlled trial to study the effects of Kimanya-Ngeyo's teacher training program on student outcomes in teachers working at 39 secondary schools in rural Uganda. For question 1), this study has innovated new measures of the teacher’s understanding of the purpose of the pedagogical approach they adopt in the classroom. Regarding this, we ask (Q1.1) is there internal alignment in the relationship between a teacher’s understanding of the objectives they are carrying out and their actual pedagogical acts across time? (Q1.2) is internal alignment reflected in students’ experience of a teacher’s pedagogy (external alignment)? (Q1.3) does this purpose-practice alignment drive learning outcomes in students?Question 2) answers three key research questions related to replication in secondary schools: (Q2.1) Does the program replicate in secondary schools? Does the program effect remain high when teachers do not self-select into training? (Q2.2) Are there teacher-to-teacher spillovers? (Q2.3) Does training with a friend improve program effectiveness?
External Link(s)

Registration Citation

Citation
El-Kashlan, Moustafa et al. 2023. "Purpose, Practice and Teacher Transformation - An RCT of a “Learning to Learn” Teacher Training in Ugandan Secondary Schools." AEA RCT Registry. November 01. https://doi.org/10.1257/rct.12347-1.0
Experimental Details

Interventions

Intervention(s)
The teacher training program is implemented by Kimanya-Ngeyo Foundation for Science and Education (KN), an organization founded in 2007 that has offered teacher training in primary schools since 2014 and in secondary schools since 2021. KN uses a curriculum titled Preparation for Social Action, which has been in development for over 40 years by Fundación para la Aplicación y Enseñanza de la Ciencia (FUNDAEC), whose aim was to integrate the traditional knowledge of rural populations in Colombia with modern scientific epistemological approaches to advance autochthonous and local processes of development. FUNDAEC has trained organizations across the world to study its workbooks using a pedagogical approach closely linked to a well-articulated conceptual framework. KN’s approach was to select a subset of the workbooks to study with teachers alongside FUNDAEC’s description of its own conceptual framework. This results in teachers having an experience in which they are treated as “``learners'' within a learning experience where the pedagogy, curriculum and context are all very carefully and deliberately tied to a well-thought out and explicitly articulated philosophical approach and conceptual framework.

As a result, the training is holistic and multi-faceted. As a first step, the training uses a carefully curated workbook to facilitate group study, read-aloud techniques accompanied by discussion, and practical learning activities. Each workbook is intended to help participants develop capacity to think and act consciously in a specific domain of activity: e.g., in describing the world around them with greater clarity, in analyzing the causes of health and disease, in making more conscious technological choices in the domain of agriculture, and more. The approach leads teachers to see themselves as scientists with a vision of generating and applying knowledge that leads to community transformation. Teachers learn to use precise language to observe and describe their communities, to pose sharp questions, to frame specific hypotheses, to use evidence and data gathered from everyday life to analyze the world around them. Practically speaking, the materials can challenge the teachers to develop these capabilities through the use of simple activities - such as as creating shapes out of clay to learn only to describe the same shapes with greater precision - to the use of complex activities that create conceptual connections across different domains of knowledge - such as measuring the point at which water changes from liquid to gas and connecting the concept of change of phase to transformation of a child who undergoes education.

A second step analyzes the teachers’ own experiences in the training through a meta-cognitive lens. They are asked to analyze the curricular and pedagogical features of the training that led them to see themselves in this new light (as scientists). We define the banking approach to education as in (Freire, 1970) as an approach to education where its primary focus is to “fill” students with knowledge which they acquire information without requiring to use critical thinking. We will refer to the opposite of this approach as a capabilities approach to education, where students are encouraged to develop their capacity for scientific thinking and the use of precise language in order to more accurately describe their reality and become active participants in their own learning process. This intervention is often the first time teachers begin to identify the difference between conceptual understanding - a necessary feature of a capabilities approach - and information assimilation - the dominant activity under a banking approach. Teachers also begin to recognize the importance of humility in being able to see things through the student’s perspective and becoming more attentive to a holistic set of qualities which characterize the student - i.e., beyond defining and comparing students according to their academic merits alone.

Many teachers report feeling uncomfortable with the format at first because they do not immediately see its connection to the approaches they would take as teachers. As they continue to participate in the training, however, they begin to see how it reshapes their orientation to the purpose of education and knowledge itself. The experiences they acquire as learners give them examples of pedagogical approaches that they initially emulate in their own classrooms. The meta-cognitive analysis of these experiences helps teachers act in a self-reflexive manner to eventually learn how to adapt these experiences through the analysis of their use of new pedagogical approaches in their classrooms.

The content of the training is not substantively different relative to the primary training which is described with more detail in Nourani et al (2020).
Intervention Start Date
2021-11-29
Intervention End Date
2025-02-01

Primary Outcomes

Primary Outcomes (end points)
Teacher level
- Pedagogy/classroom activities
- Purpose/objective of the classes (internal to the teacher)
- Internal alignment between activities and purposes (internal to the teacher)
- Measure of teachers’ “Intertemporal Alignment” strategies to transform pedagogy
- External alignment between activities and purposes (teacher-student)
- Motivation (love for the profession)
- Precision of language (using language related to pedagogy coherently and consistently)
- Teachers’ Influence on Other Teachers in School
- Influence of the school in the teaching practices (meta-alignment)
- Epistemological beliefs of the teacher
- Teacher’s Pperception of students

Student level
- Student content knowledge, application skills, and critical thinking
- Standardized tests
- Precision of Language
- Motivation to Attend School
- Epistemological beliefs of the students
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
This is a randomized controlled trial. The design is as follows. Prior to randomization, a set of 640 eligible teachers comprising about 60% of teachers in each school was identified. Then eligible teachers were each sorted into one of four “cliques.” Cliques were chosen such that self-reported social network ties within groups were maximized. Then eligible teachers were sorted again into each of four “anticliques” where social ties within anticliques were minimized.

Randomization then occurred in two stages. First, 39 schools were sorted into 13 triples of three schools each. Within each triple one school was selected at random for “Cliques” treatment, one school was selected for “Anticliques” treatment, and the last school in the triple was left as a pure control. The second stage of randomization was at the teacher level. For “Clique” schools, one of the four cliques of teachers was chosen at random to be invited to training in year 1, a second clique in year 2, a third clique in year 3, and the final clique was left as a pure control. For the “Anticlique” schools, one of the four anticliques of teachers was chosen at random to be invited to training in year 1, a second anticlique in year 2, a third anticlique in year 3, and the final anticlique was left as a pure control. Regardless of whether a school was chosen for clique or anticlique the probability of any eligible teacher being invited to training in a given year was 50%.

The experimental design also includes a plan for teacher attrition. Not every invited teacher will attend. Prior to randomization all of the eligible teachers in each school who were not assigned to a clique (anticlique) were assigned as backups to one clique (anticlique). The backups were ranked. If a teacher in a clique (anticlique) was invited to training but was no longer able to attend, the highest ranked backup for that clique (anticlique) would receive an invitation.
Experimental Design Details
Not available
Randomization Method
Randomization was done in the office by a computer on November 29, 2021 and the intervention started one day after.
Randomization Unit
Randomization was in two stages. The first stage randomized at the school level. The second stage randomized within schools at the teacher level by teacher cluster.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
Treatment (an invitation to KN training) was randomized first at the school level and then at teacher level. There were 39 schools so this makes 39 school-level treatment clusters. At the teacher level, teachers in treatment schools were invited to train in clusters and there were exactly four clusters per school. This makes 156 teacher clusters conditional on school-level treatment.
Sample size: planned number of observations
We have thus far collected baseline data in 2021, followed by a midline survey in 2022 and are currently collecting the second midline (October 2023). We plan to collect our endline survey in 2024. Our baseline survey was disrupted by a Covid-19 lockdown in Uganda, so our data collection is split between in-person and phone surveys. For the aforementioned midline and endline surveys in 2022, 2023 and 2024, we intend to interview all lower secondary teachers in the 39 study schools (around 1,000 each survey round) and 10 students per school-class-stream (between 1,500 and 2,000 per year). Moreover, we conduct assessments with all the students from S1 to S4 in 2022, 2023 and 2024 (around 20,000 observations per year). Finally, we have access to the Uganda Certificate of Education (UCE) Results (the Ugandan equivalent to an O-Level examination) by students for all the schools in our sample (around 4,000 observations (only S4) per year). We will continue to collect UCE results from the Uganda National Examinations Board after 2024.
Sample size (or number of clusters) by treatment arms
We will interview a planned 2,000 students and 640 teachers per year in 2022, 2023, and 2024 evenly spread among 13 pure control schools, 13 anticlique schools and 13 clique schools.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
University of Chicago
IRB Approval Date
2023-04-24
IRB Approval Number
IRB23-0497
IRB Name
The Mildmay Uganda Research and Ethics Committee
IRB Approval Date
2021-03-24
IRB Approval Number
0510-2017