Culturally and context specific student assessment

Last registered on March 08, 2021

Pre-Trial

Trial Information

General Information

Title
Culturally and context specific student assessment
RCT ID
AEARCTR-0007309
Initial registration date
March 08, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
March 08, 2021, 10:23 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Primary Investigator

Affiliation

Other Primary Investigator(s)

PI Affiliation
PI Affiliation
PI Affiliation
PI Affiliation
PI Affiliation

Additional Trial Information

Status
In development
Start date
2021-03-15
End date
2021-06-30
Secondary IDs
Abstract
This project explores the effectiveness of culturally and contextually relevant assessment tasks in primary and secondary education.
External Link(s)

Registration Citation

Citation
Dobrescu, Loretti et al. 2021. "Culturally and context specific student assessment." AEA RCT Registry. March 08. https://doi.org/10.1257/rct.7309
Experimental Details

Interventions

Intervention(s)
We estimate the effect of adapting and deploying a universal test similar to the National Assessment Program – Literacy and Numeracy (NAPLAN) test to children from different cultures and circumstances.

The sample will include a minimum of (i) 240 Aboriginal and Torres Strait Islander (ATSI) students currently in year 6 (11-12 years old) and their class peers, and (ii) 240 ATSI students currently in year 8 (14-15 year old) and their class peers. In each group, half of the student will be randomly assigned into the group receiving the standard test, whilst the other will receive the contextualized test.

The goal of this project is to explore whether students are more responsive and engaged in assessment tasks if they are locally contextualized and, importantly, whether students perform better on standardized assessments if these are made relevant to their own backgrounds and cultural competencies. We are additionally investigating the mechanism behind these effects.
Intervention Start Date
2021-03-22
Intervention End Date
2021-03-26

Primary Outcomes

Primary Outcomes (end points)
1. Differences in test performance between students who undertook the standard tests compared to those who undertook the contextualized tests.
2. Differences in perceived test effort between students who undertook the standard tests compared to those who undertook the contextualized tests.
3. Differences in perceived test relevance between students who undertook the standard tests compared to those who undertook the contextualized tests.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Important details:
• There are 2 experimental groups: one control and one treatment.
• In control ("Standard Test"), students will receive a standard numeracy and reading test (in the same style as NAPLAN).
• In treatment ("Culturally Appropriate Test"), students will receive a revised version of the standard test that will take into account their context and cultural background.
• There are no differences in the educational content of the two tests (just merely in their form) and each of the numeracy and reading tasks will take 50 minutes regardless of their (standard or contextualised) form.
Experimental Design Details
Randomization Method
Randomization will be carried out by a computer.
Randomization Unit
Randomization will be assigned at the individual (student) level.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
NA
Sample size: planned number of observations
The analysis involving main outcomes (1) is based on (i) 240 ATSI students currently in year 6 and 500 of their class peers, and (ii) 240 ATSI students currently in year 8 and 500 of their class peers. Depending on the effort indicator mentioned in main outcomes (2-3), we will have the same number of observations as above.
Sample size (or number of clusters) by treatment arms
We have one treatment ("Culturally Appropriate Test") and one control (“Standard Test”), across 4 groups:
1. ATSI students in year 6: 120 treatment and 120 control
2. ATSI students in year 8: 120 treatment and 120 control
3. Peers of ATSI students in year 6: 250 treatment and 250 control
4. Peers of ATSI students in year 8: 250 treatment and 250 control
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We conduct power calculations using the 2019 NAPLAN scores for the relevant groups. We assume power of 0.80 and significance level of 0.05. For test scores, we are powered for the following minimum detectable effect sizes: • ATSI year 6 reading: 25 points; 0.36 standard deviations; 5.5% • ATSI year 6 numeracy: 21 points; 0.36 standard deviations; 4.8% • ATSI year 8 numeracy: 25 points; 0.36 standard deviations; 5.0% • Peers of ATSI year 6 reading: 17 points; 0.25 standard deviations; 3.4% • Peers of ATSI year 6 numeracy: 17 points; 0.25 standard deviations; 3.4% • Peers of ATSI year 8 numeracy: 20 points; 0.25 standard deviations; 3.5%
IRB

Institutional Review Boards (IRBs)

IRB Name
Human Research Ethics Committee of the University of New South Wales
IRB Approval Date
2020-09-23
IRB Approval Number
HC200692
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials