|
Field
Trial Start Date
|
Before
July 01, 2023
|
After
September 01, 2023
|
|
Field
Trial End Date
|
Before
December 31, 2023
|
After
September 01, 2024
|
|
Field
Last Published
|
Before
January 09, 2023 11:04 PM
|
After
June 30, 2023 12:50 PM
|
|
Field
Intervention Start Date
|
Before
July 01, 2023
|
After
September 01, 2023
|
|
Field
Intervention End Date
|
Before
December 31, 2023
|
After
June 30, 2024
|
|
Field
Planned Number of Clusters
|
Before
240 School Counselors
|
After
1300 School Counselors
|
|
Field
Planned Number of Observations
|
Before
960 transcript ratings over 240 School Counselors
|
After
5232 transcript ratings over 1308 School Counselors
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
Blinded transcripts without PSAT scores - 60 school counselors
Blinded transcripts with PSAT scores - 60 school counselors
Race/gender transcripts without PSAT scores - 60 school counselors
Race/gender transcripts with PSAT scores - 60 school counselors
|
After
Blinded transcripts without PSAT scores - 65 counselors (260 transcripts)
Blinded transcripts with PSAT scores - 65 counselors (260 transcripts)
Race/gender transcripts without PSAT scores - 589 counselors (2356 transcripts)
Race/gender transcripts with PSAT scores - 589 counselors (2356 transcripts)
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
Preparedness Rating (value 1-10):
Minimal Detectable Effect Size: 0.8 points (from pilot study'0
Percentage: 80%
Alpha: 0.05
two-tailed comparison
In order to capture the 3 levels of SES, the sample size would need to be 876 transcript ratings.
However, with 4 treatments per survey (and assuming 876 as our goal), that would require distribution of 219 surveys.
We are aiming for 240 completed surveys.
|
After
Using a participant-level randomized cluster, and treatment at level 2, we conducted a power calculation to determine the number of clusters needed to have a sufficiently powered study. We used a standard alpha of .05. Estimated effect size was drawn directly from the pilot study (Francis, de Oliveira, and Dimmitt, 2019). Due to the overall goals of the future study, we used the effect size of Black Female students in assignment to AP coursework, which had an effect size of 0.2. The calculated variance in effect size was 0.32. The r-squared was also pulled directly from the pilot study at 0.249. The sample size is determined by the number of tests within the cluster, each cluster has four experimental evaluations (not counting the two baseline evaluations). Using Optimal Design software, this presented us with a needed sample size of 218 evaluations per hypothesis. With 2 racial groups, 2 genders, 3 SES levels, and 2 statistical discrimination groupings, we have 24 total hypotheses (2x2x3x2) for our main analysis. The required number of evaluations would be 5,232. Each counselor will conduct four experimental evaluations for a total of 1,308 counselors required.
|
|
Field
Did you obtain IRB approval for this study?
|
Before
No
|
After
Yes
|