AI Policy in Education - A Randomized Controlled Trial

This study evaluates the impact of different AI usage policies on secondary school students' academic performance and learning behaviors through a randomized controlled trial (RCT). The aim is to provide insights into how various AI policies can optimize student learning outcomes.

The intervention is structured into four groups with varying AI usage policies:

Group A (Control Group): No AI usage guidelines are applied. Students in this group have unrestricted access to AI tools, allowing them to use AI freely for their learning without any imposed limits or monitoring.

Group B (Treatment 1): This group receives AI guidelines, but students still have free access to AI tools. The guidelines provide general suggestions on how to use AI for learning but do not impose strict limitations on usage, allowing students to apply AI according to their personal preferences.

Group C (Treatment 2): In this group, AI use is conditional. Students are allowed to use AI tools, but their usage is capped at 30% of their total learning activities. This limitation ensures that students do not overly rely on AI for all aspects of their academic work. The purpose is to foster a balanced approach, where AI is used as a supplemental tool rather than a primary resource.

Group D (Treatment 3): Similar to Group B, this group has no specific AI usage guidelines. However, there is a 30% limit on AI use, just like Group C. This is intended to restrict excessive AI use, while students are free to use the AI tools as they wish, but they are still encouraged to engage in independent learning for the remaining 70% of their academic activities.

Teachers in all groups are responsible for ensuring the consistent implementation of the assigned AI policies across different subjects. The intervention lasts for one semester, from April to July 2025, with the primary goal of examining the effects of these varying AI usage policies on students' academic performance, learning behaviors, and overall educational outcomes.

2025-04-01

2025-08-01

The study focuses on key outcome variables related to the impact of AI usage policies in secondary education. These include:
Academic Performance – Changes in student learning outcomes and assessment results.
Learning Behavior – Variations in study habits, engagement, and reliance on AI tools.
Independent Learning Ability – The extent to which students develop critical thinking and problem-solving skills.
Teacher Instructional Approaches – Adjustments in teaching strategies and classroom dynamics in response to AI integration.

This study employs a randomized controlled trial (RCT) to evaluate the impact of different AI usage policies in secondary education. Participating classes are randomly assigned to varying AI policy conditions, allowing for a systematic comparison of their effects on student learning and teaching practices. The experiment examines how different approaches to AI integration influence academic performance, learning behaviors, and instructional methods. By analyzing these outcomes, the study aims to provide evidence-based insights for policymakers and educators on the responsible and effective use of AI in education.

Randomization will be conducted using a computer-generated algorithm to ensure an unbiased assignment of participants. Classes will be randomly allocated to different AI policy conditions through stratified randomization, considering factors such as grade level and baseline academic performance. This approach helps balance key characteristics across groups, enhancing the reliability of the study’s findings.

Class level

Yes

30 classes

1200 students

8 classes for control, 22 classes for treatment