x

The AEA RCT Registry will be down for maintenance on Tuesday, April 20th, from 9pm EDT to 11pm EDT to perform necessary upgrades. The site will be in maintenance mode during that time. We apologize for any inconvenience this may cause.
Culturally and context specific student assessment
Last registered on March 08, 2021

Pre-Trial

Trial Information
General Information
Title
Culturally and context specific student assessment
RCT ID
AEARCTR-0007309
Initial registration date
March 08, 2021
Last updated
March 08, 2021 10:23 AM EST
Location(s)

This section is unavailable to the public. Use the button below to request access to this information.

Request Information
Primary Investigator
Affiliation
Other Primary Investigator(s)
PI Affiliation
PI Affiliation
PI Affiliation
PI Affiliation
PI Affiliation
Additional Trial Information
Status
In development
Start date
2021-03-15
End date
2021-06-30
Secondary IDs
Abstract
This project explores the effectiveness of culturally and contextually relevant assessment tasks in primary and secondary education.
External Link(s)
Registration Citation
Citation
Dobrescu, Loretti et al. 2021. "Culturally and context specific student assessment." AEA RCT Registry. March 08. https://doi.org/10.1257/rct.7309-1.0.
Experimental Details
Interventions
Intervention(s)
We estimate the effect of adapting and deploying a universal test similar to the National Assessment Program – Literacy and Numeracy (NAPLAN) test to children from different cultures and circumstances.

The sample will include a minimum of (i) 240 Aboriginal and Torres Strait Islander (ATSI) students currently in year 6 (11-12 years old) and their class peers, and (ii) 240 ATSI students currently in year 8 (14-15 year old) and their class peers. In each group, half of the student will be randomly assigned into the group receiving the standard test, whilst the other will receive the contextualized test.

The goal of this project is to explore whether students are more responsive and engaged in assessment tasks if they are locally contextualized and, importantly, whether students perform better on standardized assessments if these are made relevant to their own backgrounds and cultural competencies. We are additionally investigating the mechanism behind these effects.
Intervention Start Date
2021-03-22
Intervention End Date
2021-03-26
Primary Outcomes
Primary Outcomes (end points)
1. Differences in test performance between students who undertook the standard tests compared to those who undertook the contextualized tests.
2. Differences in perceived test effort between students who undertook the standard tests compared to those who undertook the contextualized tests.
3. Differences in perceived test relevance between students who undertook the standard tests compared to those who undertook the contextualized tests.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
Important details:
• There are 2 experimental groups: one control and one treatment.
• In control ("Standard Test"), students will receive a standard numeracy and reading test (in the same style as NAPLAN).
• In treatment ("Culturally Appropriate Test"), students will receive a revised version of the standard test that will take into account their context and cultural background.
• There are no differences in the educational content of the two tests (just merely in their form) and each of the numeracy and reading tasks will take 50 minutes regardless of their (standard or contextualised) form.
Experimental Design Details
Not available
Randomization Method
Randomization will be carried out by a computer.
Randomization Unit
Randomization will be assigned at the individual (student) level.
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
NA
Sample size: planned number of observations
The analysis involving main outcomes (1) is based on (i) 240 ATSI students currently in year 6 and 500 of their class peers, and (ii) 240 ATSI students currently in year 8 and 500 of their class peers. Depending on the effort indicator mentioned in main outcomes (2-3), we will have the same number of observations as above.
Sample size (or number of clusters) by treatment arms
We have one treatment ("Culturally Appropriate Test") and one control (“Standard Test”), across 4 groups:
1. ATSI students in year 6: 120 treatment and 120 control
2. ATSI students in year 8: 120 treatment and 120 control
3. Peers of ATSI students in year 6: 250 treatment and 250 control
4. Peers of ATSI students in year 8: 250 treatment and 250 control
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We conduct power calculations using the 2019 NAPLAN scores for the relevant groups. We assume power of 0.80 and significance level of 0.05. For test scores, we are powered for the following minimum detectable effect sizes: • ATSI year 6 reading: 25 points; 0.36 standard deviations; 5.5% • ATSI year 6 numeracy: 21 points; 0.36 standard deviations; 4.8% • ATSI year 8 numeracy: 25 points; 0.36 standard deviations; 5.0% • Peers of ATSI year 6 reading: 17 points; 0.25 standard deviations; 3.4% • Peers of ATSI year 6 numeracy: 17 points; 0.25 standard deviations; 3.4% • Peers of ATSI year 8 numeracy: 20 points; 0.25 standard deviations; 3.5%
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Human Research Ethics Committee of the University of New South Wales
IRB Approval Date
2020-09-23
IRB Approval Number
HC200692
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information