Increasing diversity identification in the Australian Public Service

Last registered on October 04, 2023

Pre-Trial

Trial Information

General Information

Title
Increasing diversity identification in the Australian Public Service
RCT ID
AEARCTR-0011854
Initial registration date
October 02, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 04, 2023, 4:55 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Behavioural Economics Team of the Australian Government

Other Primary Investigator(s)

PI Affiliation
BETA
PI Affiliation
BETA
PI Affiliation
BETA

Additional Trial Information

Status
Completed
Start date
2023-06-28
End date
2023-07-13
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
In partnership with the Australian Public Service Commission, BETA is exploring ways to increase rates of disability identification in APS agency HR systems.
In partnership with a large Australian Public Service (APS) Agency who was planning to send reminder emails to staff to update their diversity information, BETA designed and conducted a four-arm trial to explore the effect of email reminders on diversity identification.
This research will test four different email reminders: three different behaviorally informed emails against attention control which is standard, information only BAU email.
Findings from this research may inform emails used in this APS agency and inform work undertaken in other APS agencies.
External Link(s)

Registration Citation

Citation
Kyaw-Myint, Su Mon et al. 2023. "Increasing diversity identification in the Australian Public Service." AEA RCT Registry. October 04. https://doi.org/10.1257/rct.11854-1.0
Experimental Details

Interventions

Intervention(s)
The APS Agency staff will be randomly allocated into one of the four groups below and BETA will compare the outcome variables for each group to determine the impact of the intervention.
1. Simplified BAU T1+email (attention control)
This email provides a basic reminder and simple information about the need to update ones details in the HR System.
2. ‘Make it Easy’ email (treatment group 1)
This email highlights the ease of updating demographic data. This draws on the EAST principle (The Behavioural Insights Team) of ‘Make it Easy’, where removing small frictions that make a task seem challenging or effortful, can increase uptake of a behaviour.
3. ‘Make it Attractive’ email (treatment group 2)
This email is designed to attract peoples’ attention and make updating details appear more rewarding (by highlighting the benefits of keeping HR System information up-to date).
4. ‘Make it Social’ email (treatment group 3)
This email highlights how the APS Agency is progressing towards a more diverse and inclusive workplace. This draws on the EAST principle of ‘Make it Social’, which recognises that humans are positively influenced by those around them and are encouraged by the behaviour of others, particular in uncertain situations.
This email will graphically show how more staff are reporting in the APS Agency HR system than ever before. This information will act as a call to action and could reflect that trust in the agency is growing over time. The graph is also visually stimulating, novel and demonstrates how the APS Agency is using staff data, which could encourage more staff to identify in the HR system.
Intervention (Hidden)
Intervention Start Date
2023-06-28
Intervention End Date
2023-07-13

Primary Outcomes

Primary Outcomes (end points)
The primary outcomes for this trial proportion of staff reporting as having a disability AND proportion of staff reporting as having ‘other diversity’ status.
Primary Outcomes (explanation)
1. Proportion of staff reporting as having a disability (disability = Yes)
2. Other diversity status. This will capture diversity status other than disability. It will be the proportion of APS Agency staff who indicated on the APS Agency’s HR system as either:
a. LGBTIQ+ (LGBTQ+ = Yes) or
b. Neurodivergent (Neurodivergent = Yes) or
c. First Nations (First Nations = Yes) or
d. Culturally and Linguistically Diverse (CALD) (CALD = Yes)

Secondary Outcomes

Secondary Outcomes (end points)
There will be two secondary outcomes (average proportion of missing data by treatment arm & average proportion of people who changed their responses from ‘Choose Not to Give’ to either ‘Yes’ or ‘No’).
Secondary Outcomes (explanation)
1. Proportion of missing data on all diversity variables. This will examine whether APS Agency staff go and update their diversity status after the trial emails. This will consist of all responses other than yes or no. Individual proportion will be created to determine proportion missing per person (out of 5 diversity indicators). This means if a person has two missing data out of 5 diversity indicators, they will get a score of 0.4. Individual level outcomes will be averaged within each arm of the trial to obtain the average proportion of missing data by arm. This will provide a measure of engagement from the reminder emails.
2. Proportion of people who changed their responses from “Choose Not to Give” to either ‘Yes’ or ‘No’ for all diversity variables. Each person will be recorded a change if they change from ‘Choose Not to Give” to either ‘Yes’ or ‘No’. So each person will have a change score created out of the five diversity indicators. They will get a score of 0.2 if they made this change for 1 out of 5 diversity indicators. Individual level outcomes will be averaged within each arm of the trial to obtain the average proportion of people who made this change by arm.

Experimental Design

Experimental Design
This research will consist of 4-arm randomised controlled trial. Intervention is delivered via email.
Experimental Design Details
Randomization Method
Complete randomisation is conducted using the randomizer package in R.
Randomization Unit
Individual level
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
Zero
Sample size: planned number of observations
19,000 staff
Sample size (or number of clusters) by treatment arms
4700 staff per arm
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The experiment will have 80 per cent power and a family wise alpha of 0.1. Assuming the global diversity rate in the control group is 54.2% (based on historical data), we will be able to detect an increase of 2.2 percentage point increase in the treatment group. Assuming the disability identification rate in the control group is 5.0% (based on historical data), we will be able to detect 1.0 percentage point difference in disability identification rates.
IRB

Institutional Review Boards (IRBs)

IRB Name
Macquarie University Human Research Ethics Committee Humanities and Social Sciences (HREC HASS)
IRB Approval Date
2023-06-06
IRB Approval Number
15615
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials