Assessing the effectiveness of significant team-based learning application exercises relative to traditional practice problems in TBL principles of microeconomics classrooms
Last registered on January 05, 2020

Pre-Trial

Trial Information
General Information
Title
Assessing the effectiveness of significant team-based learning application exercises relative to traditional practice problems in TBL principles of microeconomics classrooms
RCT ID
AEARCTR-0005225
Initial registration date
December 30, 2019
Last updated
January 05, 2020 11:50 PM EST
Location(s)

This section is unavailable to the public. Use the button below to request access to this information.

Request Information
Primary Investigator
Affiliation
Other Primary Investigator(s)
PI Affiliation
North Carolina A & T University
PI Affiliation
Glendale Community College
PI Affiliation
Pacific University
Additional Trial Information
Status
On going
Start date
2019-09-02
End date
2020-04-30
Secondary IDs
Abstract
The purpose of this study is to assess, using a multi-site randomized controlled trial, whether in-class exercises designed based on the "4-S" principles of the team-based learning pedagogy are more effective at developing student learning in college principles of microeconomics classes than traditional principles of microeconomics practice problems in those classes. Assignment to treatment and control exercises is randomized at the site and module level, so that all students, sites and instructors are exposed to both the treatment and the control. Effectiveness is measured via a low-stakes end-of-block assessment that is common for each module across all sites.
External Link(s)
Registration Citation
Citation
Maier, Mark et al. 2020. "Assessing the effectiveness of significant team-based learning application exercises relative to traditional practice problems in TBL principles of microeconomics classrooms." AEA RCT Registry. January 05. https://doi.org/10.1257/rct.5225-1.0.
Sponsors & Partners
Sponsor(s)
Experimental Details
Interventions
Intervention(s)
Interventions are treatment and control versions of application exercises (AEs) to be used in four modules (supply & demand, firm costs & competitive markets, imperfect competition, and externalities) of principles of microeconomics courses at each study site. Treatment AEs employ the "4-S" (exercises involve a Significant problem, all teams work on the Same problem, all teams are required to make a Specific choice, and all teams report Simultaneously) methodology that is part of the Team-Based Learning (TBL) pedagogy. Control AEs are standard end-of-chapter problem-type exercises that appear in most principles of microeconomics textbooks. All treatment AEs are available for use at: https://serc.carleton.edu/econ/tbl-econ/activities.html. (There are other AEs posted on this site besides the four used for this study.) For each module, participating sections of principles of microeconomics courses will incorporate the following common materials: either a treatment or control in-class exercise and a module post-test. Treatment and control modules are randomly assigned at the site-module level, so that all students at each site are exposed to two treatment modules and two control modules.
Intervention Start Date
2019-09-03
Intervention End Date
2019-12-15
Primary Outcomes
Primary Outcomes (end points)
Each student's standardized score on each of 4 module assessment quizzes
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
(1) Fraction of members of students from under-represented groups who participated in this study who either (1) take additional courses in economics; or (b) declare economics relative to the overall fractions for each study site.
(2) The distribution of correct vs. incorrect answers on each question on the four common assessment instruments.
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
Each site is randomly assigned to two treatment modules and two control modules. At each site, class proceeds as it normally does except that for the experimental modules, students complete either the treatment or control exercise (depending on the site's random assignment for that module) and take a common low-stakes assessment of their learning for that module. Student's answers to each question on the assessment are recorded for analysis.
Experimental Design Details
Not available
Randomization Method
Assignment to treatment and control is conditionally random. Conditions to be satisfied are:
(1) Each site has two treatment modules and two control modules
(2) All sections at a site have the same assignment (either treatment or control) for a given module

Given these constraints and the fact that there are four modules, there are six possible patterns to use to randomly assign sites to a set of treatment and control modules: TTCC, CCTT, TCTC, CTCT, TCCT, CTTC. The randomization procedures was as follows:
(1) Sites are matched together so that the number of students enrolled at one site is roughly equal to the number of students enrolled at another site. For example, if one site has a projected enrollment of 35 students, a second site has a projected enrollment of 37 students, and a third site has a project enrollment of 70 students, then the matching algorithm would place sites 1 and 2 into the opposite randomization pattern as site 3.
(2) One half of each site match is randomly assigned a pattern via a computerized random number draw that simulates the roll of a 6-sided die. Once one half of the match is assigned a pattern, the other half's randomization pattern is set.
Randomization Unit
Unit of randomization is a site. Random assignment is to a pattern of 2 treatment and 2 control modules for that site.
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
Clustering is at the section level (some sites have multiple sections). Original plan was to have 8 sites and 13 total clusters. We lost one site with two sections part-way through the semester (after randomization was completed). Thus, the final number of sites is 7, with 11 total clusters.
Sample size: planned number of observations
An observation is a student-module combination. The original plan with 8 sites resulted in 1560 student-module combinations. After losing a site, it is expected that approximately 1400 student-module combinations remain.
Sample size (or number of clusters) by treatment arms
Of the 1400 expected student-module combinations, approximately 700 observations will be treatment student-module combinations, and 700 will be control student-module combinations.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Minimum detectable effect size depends on the size of the inter-cluster correlation (ICC) and assumed value of R-squared. Power calculations were conducted for small (0.1), medium (0.5) and large (0.8) values of the ICC and low (0.2) and high (0.6) values of R-squared. Minimum detectable effect size (in standard deviations) for each ICC-R-squared combination are below. Low R-Squared High R-Squared Low ICC 0.20 0.14 Medium ICC 0.45 0.32 High ICC 0.63 0.45
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
University of Richmond Institutional Review Board
IRB Approval Date
2019-08-07
IRB Approval Number
URIRB190802
IRB Name
Pacific University Institutional Review Board
IRB Approval Date
2019-05-15
IRB Approval Number
1421839-1