Experimental Evaluation of the Use of Artificial Intelligence in Building Permit Review

Last registered on October 08, 2026
View Trial History

Pre-Trial

Trial Information

General Information

Title
Experimental Evaluation of the Use of Artificial Intelligence in Building Permit Review
RCT ID
AEARCTR-0017575
Initial registration date
December 30, 2025

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 06, 2026, 7:13 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
October 08, 2026, 12:27 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country
Chile
Region

Primary Investigator

Name
Juan Díaz
Affiliation
University of Chile
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Gabriel Weintraub
PI Affiliation
Stanford University
Contact Investigator

Additional Trial Information

Status
In development
Start date
2026-10-05
End date
2027-03-31
Keywords
Other
Additional Keywords
Administrative Process Efficiency, Artificial Intelligence, Digital Government, Human–AI Collaboration.
JEL code(s)
H11, O33, D73, C93
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This paper examines how artificial intelligence can enhance administrative efficiency in public service delivery, focusing on the processing of building permit applications. We evaluate an AI-assisted system specifically designed and implemented to experimentally assess its impact on bureaucratic performance. Using a field experiment in Chile involving architects reviewing building permit applications under conditions that mimic the work of municipal reviewers, the platform integrates AI tools into the review process to support technical assessment while preserving regulatory discretion. The analysis focuses on how AI adoption affects processing times, workflow efficiency, and administrative congestion—key sources of delay that generate economic costs for the construction sector and broader welfare losses. By reducing waiting times within a highly regulated approval process, the intervention illustrates how purpose-built AI systems can improve productivity in highly congested and bureaucratic institutional environments. Finally, the study explores heterogeneous effects using reviewers’ features to assess how baseline performance conditions shape the impact of AI assistance.
External Link(s)

Registration Citation

Citation
Díaz, Juan and Gabriel Weintraub. 2026. "Experimental Evaluation of the Use of Artificial Intelligence in Building Permit Review." AEA RCT Registry. October 08. https://doi.org/10.1257/rct.17575-2.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The intervention introduces an AI-assisted review platform into the first technical review of minor works permit applications. In a controlled laboratory setting, six architects review the same 34 real applications from two Chilean municipalities: each application is reviewed by three of them with AI assistance and by the other three without it. This design allows a comparison of review times, the quality of the resulting observation reports, and the consistency of assessments across reviewers. The intervention aims to reduce administrative delays and improve productivity without compromising the quality of regulatory review.
Intervention Start Date
2026-10-07
Intervention End Date
2026-11-23

Primary Outcomes

Primary Outcomes (end points)
The primary outcome of interest is the effective time required to complete the first technical review of a minor works permit application, measured in minutes from the moment the reviewer opens the application until the observation report is submitted. It excludes pauses and, in the treatment arm, includes the time needed to load the application into the AI platform. This outcome captures a key dimension of administrative efficiency and directly reflects the delays experienced by applicants and the construction sector. Review time is recorded with the reviewer's stopwatch log and reconciled daily with the start and end times recorded by the coordinator, allowing precise comparisons between treatment and control conditions. The primary analysis estimates the effect of AI assistance on the logarithm of review time, with reviewer and application fixed effects and a one-sided test, and is interpreted jointly with the quality and consistency of the resulting reports. By focusing on review time without neglecting quality, the study evaluates whether AI tools can meaningfully accelerate permit review while preserving existing regulations and professional judgment.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary outcomes examine how AI assistance affects the quality and consistency of reviews, and how its effects vary across reviewers and permit types. Quality is measured by comparing each observation report, under blind assessment, with a reference report: the share of relevant and of substantive observations detected, observations without legal basis, and report precision. Consistency is measured by the dispersion of review times and the agreement of findings among the three reviewers who assess the same application under the same condition. Additional outcomes assess heterogeneity by reviewer, permit type, municipality, application size and review order, and are contrasted with a closing survey of reviewers. Together, these measures show whether time savings come at the expense of quality and whether AI makes reviews more uniform across professionals. All secondary outcomes are constructed from time logs, observation reports and their blind assessment, recorded identically in both arms.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The study implements a randomized controlled trial (RCT) in a laboratory setting to evaluate the causal impact of AI assistance on the first technical review of minor works permit applications from two Chilean municipalities. It compares review as currently conducted in a municipal works department, including any standard support such as other AI tools, with review using the AI platform as the preferred support. Assignment follows a crossed design with complementary pairing: each application is reviewed by all six reviewers, three with AI assistance and three without, and each reviewer handles half of the applications under each condition. This ensures exact balance across applications and reviewers, the two main sources of heterogeneity in review times. The design allows credible causal inference on the effects of AI assistance on review time, quality and consistency, while reviewers retain full authority over each observation report under existing regulations.
Experimental Design Details
Not available
Randomization Method
Randomization is implemented through a computerized procedure designed to ensure exact balance through complementary pairing. Specifically, the 34 applications are grouped into 17 fixed pairs, in an order set before the draw. For each pair, the procedure randomly selects which application comes first and one of the 20 possible subsets of three reviewers who review it with AI assistance; the complementary subset reviews the second application with AI assistance. Each reviewer's review order is then drawn as an independent random permutation, and each reviewer-application combination receives a unique blind code. The procedure uses a pseudorandom generator with fixed seeds and is replicated bit by bit by two independent implementations, ensuring reproducibility, balance and protection against discretionary assignment or manipulation. This approach supports unbiased estimation of treatment effects under standard assumptions of randomized experiments.
Randomization Unit
The unit of randomization is the individual review, defined as a reviewer-application combination. Each application is reviewed by all six reviewers, and treatment is assigned at the review level: three reviews of each application are conducted with AI assistance and three without. Every reviewer therefore evaluates both treated and control applications, and every application is reviewed under both conditions, enabling efficient estimation of treatment effects while accounting for reviewer-specific and application-specific heterogeneity.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
N/A
Sample size: planned number of observations
The sample consists of 34 real minor works applications (14 from Renca and 20 from Rancagua), each reviewed by six reviewers, for a total of 204 reviews. Applications were selected among first-review submissions recorded in the AI platform and cover the three types of minor works: extensions of up to 100 m² (18), modifications without structural changes (13) and social or low-cost housing extensions (3). Data collection takes place between October 7 and 23, 2026 (12 working days). The unit of observation is the individual review (reviewer × application), generating variation both within and across applications and reviewers.
Sample size (or number of clusters) by treatment arms
Reviews are assigned to treatment and control groups through complementary pairing within the 34 × 6 application × reviewer design. Each application receives exactly three AI-assisted and three control reviews, and each reviewer conducts exactly 17 reviews in each condition. This implies an allocation of 102 treated and 102 control reviews, fixed by design rather than dependent on realized inflow. This balanced design ensures that differences in reviewer speed and application difficulty are absorbed by reviewer and application fixed effects, so the treatment effect is estimated from within-reviewer and within-application contrasts.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Unit: individual review (reviewer × application); N = 204 reviews (34 applications × 6 reviewers). Main outcome: log effective review time, with a residual standard deviation of 0.61 log points, calibrated with minor works review times from a previous field pilot. With reviewer and application fixed effects, the crossed design yields a standard error of 0.085 for the treatment effect (σ·√(2/3N)). For a one-sided test at 5% with 80% power, the MDE is 0.21 log points (0.35 SD), equivalent to a 19.1% reduction in review time (about 13 minutes over the pilot mean of 69 minutes). Power to detect a 20% reduction is 83%; across the 90% confidence interval for σ (0.49–0.83), the MDE ranges from 15.7% to 25.1%. Inference will use randomization inference and standard errors clustered by reviewer and application.
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Comité de Ética para la Investigación con Seres Humanos de la Facultad de Economía y Negocios de la Universidad de Chile
IRB Approval Date
2025-12-29
IRB Approval Number
22/2025
Analysis Plan