What is the most effective format for delivering online training?

Last registered on December 15, 2025
View Trial History

Pre-Trial

Trial Information

General Information

Title
What is the most effective format for delivering online training?
RCT ID
AEARCTR-0015387
Initial registration date
February 14, 2025

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
February 20, 2025, 5:09 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
December 15, 2025, 10:49 PM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country
Australia
Region

Primary Investigator

Name
Australian Centre of Evaluation (ACE)
Affiliation
The Treasury
Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status
Completed
Start date
2025-01-23
End date
2025-02-20
Keywords
Firms & Productivity, Governance, Labor
Additional Keywords
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
The Australian Public Service (APS) is currently offering numerous online learning opportunities ranging in content and form. These learning opportunities are an important tool for APS officers' professional development and learning. Despite the investment in these online learning opportunities across the APS and their importance to professional development, there have been few evaluations of their effectiveness. There has not been any formal research or comprehensive assessment to measure the impact of these different online learning methods on engagement, knowledge retention, skill development, and overall professional awareness. This trial will provide insights into the effectiveness of three common online learning methods, a traditional e-learning 'click through' module, a micro-learning video and a podcast. The results of the trial will guide the development of future training initiatives, ensuring resources to develop training are allocated effectively and the overall quality and engagement with online learning is enhanced.
External Link(s)

Registration Citation

Citation
(ACE), Australian Centre of Evaluation. 2025. "What is the most effective format for delivering online training?." AEA RCT Registry. December 15. https://doi.org/10.1257/rct.15387-2.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
Intervention (Hidden)
Intervention Start Date
2025-01-23
Intervention End Date
2025-02-20

Primary Outcomes

Primary Outcomes (end points)
Training Effectiveness (learning and engagement)
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Participants will be asked to take part in an online training on randomised trials. Participants will be randomised into a control group (traditional 'click through' e-learning module), or one of two treatment groups (a video or a podcast). All training will cover similar content.
Experimental Design Details
Randomization Method
By the qualtrics software.
Randomization Unit
Individual employee of the Australian public service
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
1
Sample size: planned number of observations
600
Sample size (or number of clusters) by treatment arms
200 individuals control, 200 treatment 1, 200 treatment 2.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Macquarie University Human Research Ethics Committee
IRB Approval Date
2025-01-13
IRB Approval Number
520251868360834
Analysis Plan

Analysis Plan Documents

Pre-Analysis Plan - Training Format RCT.docx

MD5: 68445a28433bd0725877f5c6120eca32

SHA1: 2d27bc8b5b575f8341c1f5fc3e656dff55190233

Uploaded At: February 14, 2025

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
February 19, 2025, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
February 20, 2025, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
752
Was attrition correlated with treatment status?
Yes
Final Sample Size: Total Number of Observations
752
Final Sample Size (or Number of Clusters) by Treatment Arms
216 participants received the click-through module, 250 participants received the podcast and 286 participants received the video format.
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
No
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
A rapid, mixed‑methods evaluation found a short video was the most suitable format for delivering a particular online training to Australian public servants.

Purpose
Online training for professionals can be delivered in various formats. This evaluation tested the effectiveness of 3 different formats for short, introductory training:
3‑minute video
15‑minute podcast
30‑minute e‑learning module

Methods
This rapid evaluation used a ‘randomised trial’, a short survey and focus groups. For the former, over 750 public servants were randomly assigned to receive training in 1 of the 3 different formats. The training participants were then asked several questions. These questions were used to create a ‘training effectiveness index’.

Key results
The randomised trial found video and module were equally effective, and both outperformed the podcast
The survey found that 52 per cent of people preferred video and only 9 per cent preferred podcast
Focus groups also favoured video

Conclusion
The trial had some limitations. However, the ACE judged that the evidence from the evaluation – and the lower cost of the video format – meant video was the most suitable format for their purposes. The study’s limitations and purpose should be borne in mind before applying the results elsewhere.
Citation
Slaven E, Bowers P, Sheshgir S, Kennedy G, Copley S and Greenwell H (2025) The effect of online training format on engagement: a randomised trial, the Australian Centre for Evaluation, the Australian Treasury.
URL
https://evaluation.treasury.gov.au/sites/evaluation.treasury.gov.au/files/2025-12/training-format-rct-report.pdf

Reports & Other Materials