x

NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
Knowledge in Action Efficacy and Maturation Studies
Last registered on July 17, 2017

Pre-Trial

Trial Information
General Information
Title
Knowledge in Action Efficacy and Maturation Studies
RCT ID
AEARCTR-0002189
Initial registration date
July 12, 2017
Last updated
July 17, 2017 11:40 AM EDT
Location(s)
Region
Primary Investigator
Affiliation
University of Southern California Center for Economic and Social Research
Other Primary Investigator(s)
PI Affiliation
Pennsylvania State University
PI Affiliation
Gibson Consulting Group, LLC
Additional Trial Information
Status
On going
Start date
2016-04-25
End date
2019-12-31
Secondary IDs
Abstract
Globalization and technological innovation compound the need for schools to prepare students for college, career, and citizenship in the 21st century. To transfer their skills, knowledge, and attitudes to new contexts, students must learn deeply to develop the ability to think and communicate in sophisticated ways, demonstrate creativity and innovation, and learn adaptively. Yet learning course content through the traditional, and currently predominant, “transmission” mode of instruction – in which teachers lecture while students take notes, memorize content, and restate content in multiple-choice assessments – is not an effective way to learn deeply. Through the Knowledge in Action (KIA) project-based approach to Advanced Placement (AP) teaching and learning, students actively engage in teacher- and student-posed learning challenges (i.e., projects) rather than playing a passive role. The goals of the KIA Efficacy Study are to 1) test the hypothesis that the KIA project-based learning approach to AP U.S. Government and AP Environmental Science leads to improved or “deeper” student learning, interpersonal and intrapersonal skills, and civic engagement and 2) identify factors that correlate with successful KIA implementation, such as support from districts, schools, the KIA professional development provider, and professional learning communities. The KIA Efficacy Study examines the impact of a one-year intervention. The KIA Maturation Study evaluates the impact of a second year of KIA.
External Link(s)
Registration Citation
Citation
Morgan, Kari, Amie Rapaport and Anna Saavedra. 2017. "Knowledge in Action Efficacy and Maturation Studies ." AEA RCT Registry. July 17. https://doi.org/10.1257/rct.2189-1.0.
Former Citation
Morgan, Kari et al. 2017. "Knowledge in Action Efficacy and Maturation Studies ." AEA RCT Registry. July 17. http://www.socialscienceregistry.org/trials/2189/history/19506.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
KIA is a response carefully designed to meet the challenges of traditional AP instruction. The initiative was based on the potential, yet under-realized, impact AP courses could have on deeper learning for all students. KIA values both transmission and inquiry instructional approaches, but seeks to maximize the two methods’ respective benefits. The KIA treatment for AP U.S. Government and AP Environmental Science in a teacher’s first year includes access to: curriculum materials through an online portal, a four-day Summer Institute, four full-day professional development sessions offered throughout the school year, one-on-one virtual coaching, and the KIA in-person and virtual professional learning community. In a teacher’s second year, the KIA treatment includes all these elements, save for one-on-one coaching, available to teachers only in their first year.
Intervention Start Date
2016-06-20
Intervention End Date
2018-05-30
Primary Outcomes
Primary Outcomes (end points)
KIA Efficacy Study outcomes include:
• Whether students take the May 2017 AP U.S. Government or AP Environmental Science examinations
• Students’ overall and sub-section scores on the AP U.S. Government or AP Environmental Science examinations
• Students’ overall and sub-section scores on the College and Work Readiness Assessment
• Survey measures of students’
a. Interpersonal skill (e.g., collaboration, communication)
b. Intrapersonal skills (e.g., self-regulation, self-efficacy)
c. Civic engagement (e.g., political efficacy, concern for the environment)

KIA Maturation Study outcomes include:
• Whether students take the May 2018 AP U.S. Government or AP Environmental Science examinations
• Students’ overall and sub-section scores on the AP U.S. Government or AP Environmental Science examinations
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
We recruited five large urban school districts geographically spread across the United States to participate in the study. Within districts, we offered study-enrollment opportunities to AP U.S. Government and AP Environmental Science teachers with at least one year of experience teaching their course (or similar prior experience). In three districts, participation was open to teachers of either AP U.S. Government and AP Environmental Science. In the fourth district, only AP Environmental Science teachers could participate, while in the fifth district the opportunity was limited to teachers of AP U.S. Government.

In the KIA Efficacy Study, across the five districts we randomized 76 schools into treatment or control arms, blocked by school district. Treatment teachers received KIA intervention starting in the summer of 2016 through May 2017, and control teachers continued to teach their AP U.S. Government or AP Environmental Science courses as usual. Students began their AP U.S. Government or AP Environmental Science course starting in September 2016.

Critical to the entire endeavor’s success was the delayed-entry design that provided all teachers enrolled in the Efficacy Study with the opportunity to use KIA curriculum and professional-development supports, either during the 2016-17 or 2017-18 school years. District research review boards likely would not have approved a design in which half the teachers never had the opportunity to use KIA resources yet still were expected to participate in research activities. Teachers may have been hesitant to enroll with the expectation of only probabilistic odds of receiving the opportunity to participate in KIA.

In the KIA Maturation Study, we are following the same teachers who participated in the KIA Efficacy Study and are teaching AP U.S. Government or AP Environmental Science in their district during the 2017-18 school year, as well as their 2017-18 AP U.S. Government or AP Environmental Science students. During this period, both groups of teachers are offered access to the intervention (i.e., the original experimental arm receives a second year of treatment and the original control arm receives a first year of treatment). We then indirectly estimate the two-year effect by summing the estimated one-year effect (from the Efficacy Study period) and the estimated difference between the first year and the second (from the Maturation Study period).
Experimental Design Details
Randomization Method
We conducted randomization by a computer using random seeds. We randomly assigned schools to two arms, blocked by district, and employed re-randomization to ensure the arms were balanced on the number of teachers by course subject (AP U.S. Government or AP Environmental Science) and covariates within each district. Allocations that were unbalanced according to predefined criteria were re-randomized.
Randomization Unit
Schools
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
76 schools
Sample size: planned number of observations
2,052 students, assuming an average of 27 students per teacher/school
Sample size (or number of clusters) by treatment arms
38 schools per each arm, resulting in 45 teachers in the experimental arm and 44 teachers in the control arm.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering) At the time of randomization, our study is powered to detect an effect of about 0.15 standard deviations (SD) assuming an alpha of 0.05, a power of 0.8, an intra-class correlation of 0.2, and a level-2 R-squared of 0.9. Decreasing the R-squared to 0.75 leads to an MDE of 0.18 SD.
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
University of Southern California University Park Institutional Review Board
IRB Approval Date
2016-05-19
IRB Approval Number
Exempt (Study #UP1600291)
Analysis Plan
Analysis Plan Documents
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports, Papers & Other Materials
Relevant Paper(s)
REPORTS & OTHER MATERIALS