Decision Support for Human Trafficking Case Prioritization in Brazil: a Cluster-Randomized Controlled Tria

Last registered on September 12, 2024

Pre-Trial

Trial Information

General Information

Title
Decision Support for Human Trafficking Case Prioritization in Brazil: a Cluster-Randomized Controlled Tria
RCT ID
AEARCTR-0014273
Initial registration date
August 29, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 12, 2024, 5:04 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
Stanford University

Other Primary Investigator(s)

PI Affiliation
Stanford University
PI Affiliation
Stanford University
PI Affiliation
Stanford University

Additional Trial Information

Status
In development
Start date
2024-10-01
End date
2026-07-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This document outlines the analysis plan for a cluster-randomized controlled trial of the effect of Bússola I.A., a human trafficking case management prioritization support tool on labor trafficking case detection in Brazil. The project is part of ``Comprehensive Action towards Forced labor Eradication (CAFE).'' The experiment will test whether federal labor prosecutors using the decision support tool to prioritize cases for task force investigations yields improved task-force-related outcomes. This pre-analysis plan provides intervention details, methodology, and a plan for analyzing the results of the experiment.
External Link(s)

Registration Citation

Citation
Babiarz, Kimberly et al. 2024. "Decision Support for Human Trafficking Case Prioritization in Brazil: a Cluster-Randomized Controlled Tria." AEA RCT Registry. September 12. https://doi.org/10.1257/rct.14273-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The proposed intervention Bússola I.A. aims to use state-of-the-art machine learning for the task of
processing incoming labor trafficking tips and complaints, automating and expediting processes that
were previously conducted manually. Figure 1 illustrates the theory of change, in which features of
Bússola I.A. address key bottlenecks hindering optimal anti-trafficking response, leading to process
improvements that have direct implications for MPT outputs and trafficking related outcomes. The
tool will deliver contextually relevant information to labor prosecutors to help prioritize hundreds of
pending trafficking tips and swiftly identify the presence of pertinent information in an incoming tip,
thus facilitating informed decision-making.

Bússola I.A. is a modular tool with features that fall into three categories. First, Bússola I.A. performs
near real time risk scoring to identify and prioritize the tips and cases most likely to lead to trafficking
discovery. To do so, the model uses a nimble, locally implemented open source large language
model to process the information contained in the incoming tip, which is submitted in free text
narrative format. The tool organizes extracts key features from tip received that are relevant for
decision making, organizing these features into a structured dataset (for example: whether violence is
reported, whether children are mentioned, whether an address is provided, economic sector). These
features are then used in a machine learning model to estimate the likelihood of trafficking discovery and the priority level of the tip. This method aims to reduce reliance on subjective intuitions or
availability-bias in the early stages of case prioritization, leading to more effective case prioritization
and improved task force targeting.

Second, Bússola I.A. uses a locally implemented large language model to produce a concise text-
summary of the key features of the tip, along with the primary components driving the tip’s risk score.
The goal of the LLM-generated summaries is to provide a quick, high-level summary to prosecutors,
and to describe the main components driving the risk score corresponding to the tip. These features
are critical for quickly conveying key pieces of information in an easy-to-understand and interpret
narrative framework, as well as explaining the risk score prioritization. The LLM augmentation
is designed to expedite tip processing for faster desk investigations and more timely task force
responses.

Third, Bússola I.A. visualizes the spatial distribution of all active tips in an interactive map showing
the locations of each active case. This feature allows prosecutors examining a given case to view all
cases originating from nearby locations and facilitates the identification of clusters of tips, leading to
larger task forces which investigate more cases.

The tool places a strong emphasis on interpretability and transparency allowing for real-time interro-
gations of model outputs to build trust with human decision makers. Moreover, the Bússola I.A. tool
is designed to complement and integrate into the existing case management system MPT Digital as a
dashboard extension, minimizing changes in workflow required to access Bússola I.A. analytics. In
other words, Bússola I.A. is designed to be a new feature of the existing MPT Digital platform rather
than a completely new replacement.
Intervention Start Date
2024-10-01
Intervention End Date
2025-09-30

Primary Outcomes

Primary Outcomes (end points)
The primary outcome for this study is the number of trafficking victims rescued in a given task force.
We determine whether a particular worker is considered a ’victim rescued’ if prosecutors and labor
inspectors make a field determination that the worker should be removed from trafficking conditions
or conditions analogous to slavery, and enrolled in social safety net programs specifically established
for trafficking victims.

Because a single task force may include a number of different site inspections pertaining to different
open cases, the total number of victims rescued per task force is the sum of rescues across all
inspections i conducted as part of a given task force within a given jurisdiction.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Task force success
Task force efficiency
Task force rescues
Secondary Outcomes (explanation)
We study each of these mechanisms separately in a supplementary analysis focusing on the following
secondary outcomes:
1. Task force success
First we measure the success or positive predictive value (PPV) of a task force, defining task
force success according to the discovery of trafficking and therefore rescue of victims (and
prosecution of traffickers). Bússola I.A. may increase task force PPV by assisting prosecutors in
identifying and prioritizing the cases most likely to lead to trafficking interdiction.

2. Task force efficiency
Second, we measure the efficiency of each task force according to the rate at which cases are
investigated in each task force. Each task force may investigate one or more unique trafficking
complaints (‘cases’) together in one field operation. Bússola I.A. may increase the number of
cases per task force by highlighting clusters and related cases.

3. Task force rescues
Finally we study the number of workers assisted per successful task force, again determining
whether a worker is a victim rescued based on whether prosecutors and labor inspectors
determine (in real time) that the worker should be removed from trafficking conditions or
conditions analogous to slavery, and enrolled in social safety net programs designated for
trafficking cases. Bússola I.A. may increase the conditional mean number of rescues (conditional
on a successful task force) if the tool assists prosecutors to identify the most severe cases with
a greater number of workers trafficked).

Experimental Design

Experimental Design
Our primary analysis is on the task force level, and task forces are clustered within jurisdiction. The 23 included clusters (i.e., jurisdictions) are placed in randomization-blocks formed by whether the jurisdiction is above or below median historical case load (using the case load records for each office over the last ten years). Given the 23 clusters, the median split puts 12 in the below and 11 in the above groups. The cluster-level randomization then occurs within blocks, with treatment:control ratios of 6:6 for the below the median block and 5:6 for the above-the-median block.
Experimental Design Details
Not available
Randomization Method
Cluster randomization done before implementation by a computer
Randomization Unit
Prosecutorial jurisdictions (roughly corresponding to Brazilian states) are randomized into either a treatment or control arm.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
23 clusters
Sample size: planned number of observations
460 task forces
Sample size (or number of clusters) by treatment arms
11 treatment jurisdictions
12 control jurisdictions
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Records from task forces occurring in the past 10 years show a baseline average of 6.9 victims rescued per task force (standard deviation: 16.65).1 Our study is powered to observe a standardized effect size of .29, or an increase in average number of rescues from 6.9 to 11.9, assuming the flow of task forces per month studied follows historical averages of about 18 task forces per year per jurisdiction, and an intracluster correlation coefficient of 0.027 (computed using historical records).
IRB

Institutional Review Boards (IRBs)

IRB Name
Stanford University Research Compliance Office
IRB Approval Date
2024-08-29
IRB Approval Number
N/A
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information