Increasing diversity identification in the Australian Public Service

Last registered on October 04, 2023

View Trial History

Pre-Trial

Trial Information

General Information

Title

Increasing diversity identification in the Australian Public Service

RCT ID

AEARCTR-0011854

Initial registration date

October 02, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

October 04, 2023, 4:55 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country

Australia

Region

Primary Investigator

Name

BETA Team Registration

Affiliation

Behavioural Economics Team of the Australian Government

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Su Mon Kyaw-Myint

PI Affiliation

BETA

Contact Investigator

PI Name

Dana Leidl

PI Affiliation

BETA

Contact Investigator

PI Name

Andrew Pfieffer

PI Affiliation

BETA

Contact Investigator

Additional Trial Information

Status

Completed

Start date

2023-06-28

End date

2023-07-13

Keywords

Behavior

Additional Keywords

diversity, inclusion, identification

JEL code(s)

Secondary IDs

Prior work

This trial does not extend or rely on any prior RCTs.

Abstract

In partnership with the Australian Public Service Commission, BETA is exploring ways to increase rates of disability identification in APS agency HR systems.
In partnership with a large Australian Public Service (APS) Agency who was planning to send reminder emails to staff to update their diversity information, BETA designed and conducted a four-arm trial to explore the effect of email reminders on diversity identification.
This research will test four different email reminders: three different behaviorally informed emails against attention control which is standard, information only BAU email.
Findings from this research may inform emails used in this APS agency and inform work undertaken in other APS agencies.

External Link(s)

Registration Citation

Citation

Kyaw-Myint, Su Mon et al. 2023. "Increasing diversity identification in the Australian Public Service." AEA RCT Registry. October 04. https://doi.org/10.1257/rct.11854-1.0

Sponsors & Partners

Partner

Name

Australian Public Service Commission

Type

government

URL

https://www.apsc.gov.au/

Experimental Details

Interventions

Intervention(s)

The APS Agency staff will be randomly allocated into one of the four groups below and BETA will compare the outcome variables for each group to determine the impact of the intervention.
1. Simplified BAU T1+email (attention control)
This email provides a basic reminder and simple information about the need to update ones details in the HR System.
2. ‘Make it Easy’ email (treatment group 1)
This email highlights the ease of updating demographic data. This draws on the EAST principle (The Behavioural Insights Team) of ‘Make it Easy’, where removing small frictions that make a task seem challenging or effortful, can increase uptake of a behaviour.
3. ‘Make it Attractive’ email (treatment group 2)
This email is designed to attract peoples’ attention and make updating details appear more rewarding (by highlighting the benefits of keeping HR System information up-to date).
4. ‘Make it Social’ email (treatment group 3)
This email highlights how the APS Agency is progressing towards a more diverse and inclusive workplace. This draws on the EAST principle of ‘Make it Social’, which recognises that humans are positively influenced by those around them and are encouraged by the behaviour of others, particular in uncertain situations.
This email will graphically show how more staff are reporting in the APS Agency HR system than ever before. This information will act as a call to action and could reflect that trust in the agency is growing over time. The graph is also visually stimulating, novel and demonstrates how the APS Agency is using staff data, which could encourage more staff to identify in the HR system.

Intervention Start Date

2023-06-28

Intervention End Date

2023-07-13

Primary Outcomes

Primary Outcomes (end points)

The primary outcomes for this trial proportion of staff reporting as having a disability AND proportion of staff reporting as having ‘other diversity’ status.

Primary Outcomes (explanation)

1. Proportion of staff reporting as having a disability (disability = Yes)
2. Other diversity status. This will capture diversity status other than disability. It will be the proportion of APS Agency staff who indicated on the APS Agency’s HR system as either:
a. LGBTIQ+ (LGBTQ+ = Yes) or
b. Neurodivergent (Neurodivergent = Yes) or
c. First Nations (First Nations = Yes) or
d. Culturally and Linguistically Diverse (CALD) (CALD = Yes)

Secondary Outcomes

Secondary Outcomes (end points)

There will be two secondary outcomes (average proportion of missing data by treatment arm & average proportion of people who changed their responses from ‘Choose Not to Give’ to either ‘Yes’ or ‘No’).

Secondary Outcomes (explanation)

1. Proportion of missing data on all diversity variables. This will examine whether APS Agency staff go and update their diversity status after the trial emails. This will consist of all responses other than yes or no. Individual proportion will be created to determine proportion missing per person (out of 5 diversity indicators). This means if a person has two missing data out of 5 diversity indicators, they will get a score of 0.4. Individual level outcomes will be averaged within each arm of the trial to obtain the average proportion of missing data by arm. This will provide a measure of engagement from the reminder emails.
2. Proportion of people who changed their responses from “Choose Not to Give” to either ‘Yes’ or ‘No’ for all diversity variables. Each person will be recorded a change if they change from ‘Choose Not to Give” to either ‘Yes’ or ‘No’. So each person will have a change score created out of the five diversity indicators. They will get a score of 0.2 if they made this change for 1 out of 5 diversity indicators. Individual level outcomes will be averaged within each arm of the trial to obtain the average proportion of people who made this change by arm.

Experimental Design

This research will consist of 4-arm randomised controlled trial. Intervention is delivered via email.

Experimental Design Details

Randomization Method

Complete randomisation is conducted using the randomizer package in R.

Randomization Unit

Individual level

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

Zero

Sample size: planned number of observations

19,000 staff

Sample size (or number of clusters) by treatment arms

4700 staff per arm

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

The experiment will have 80 per cent power and a family wise alpha of 0.1. Assuming the global diversity rate in the control group is 54.2% (based on historical data), we will be able to detect an increase of 2.2 percentage point increase in the treatment group. Assuming the disability identification rate in the control group is 5.0% (based on historical data), we will be able to detect 1.0 percentage point difference in disability identification rates.

Supporting Documents and Materials

IRB