AI Predictions, Strategic Disclosure, and Optimal Coarse Bracketing

The intervention was embedded in the shipment-tracking interface of a large digital logistics platform in China. It randomized how predicted logistics information was generated and disclosed when a device queried a particular shipment. The intervention affected only the information shown to users and did not alter the shipment, delivery process, or courier operations.

The experiment varied three dimensions:

1. Prediction technology and availability. Shipment queries were assigned to a traditional prediction model, an AI maximum-probability prediction rule, an AI fastest-time prediction rule, or a no-prediction condition.

2. Route-information disclosure. Within the two AI conditions, the interface displayed either only the predicted final delivery time or the predicted final delivery time together with predicted intermediate logistics nodes. The traditional model did not generate intermediate-node predictions.

3. Display precision. Predictions were displayed using one of four randomly assigned time intervals: one day, six hours, one hour, or thirty minutes.

These dimensions produced 21 experimental arms: four traditional-model arms, sixteen AI arms defined by two AI prediction rules, two route-disclosure conditions, and four precision levels, and one no-prediction arm. Assignment was hierarchical rather than uniform across the 21 final arms.

The intervention was conducted in two phases. Phase I ran from May 3 through May 12, 2026. In this phase, the randomized precision determined the default display, but users could change the precision before confirming their choice. Phase II ran from May 13 through June 1, 2026. In this phase, users could not change the randomly assigned display precision. The remaining treatment dimensions and the underlying shipment-tracking service were unchanged across phases.

Treatment was assigned when a platform-device-shipment combination first entered the experiment and was subsequently retained for later queries of the same shipment by the same device.

2026-05-01

2026-06-01

1. User satisfaction / evaluation outcome:
Overall user rating of the package-tracking experience
Indicator for negative feedback or complaint submission
2. Tracking query behavior:
Total number of tracking queries per shipment
Average time interval between consecutive queries
3. User precision choice:
Indicator for whether the user changes the default prediction precision
Time spent by the user changing the prediction precision
Final selected precision level

The primary outcomes fall into three categories:
1. User feedback. For each eligible shipment-tracking exposure, we measure whether the user records any feedback, positive feedback, or negative feedback. We also construct an indicator for negative feedback accompanied by a specific reason explicitly related to the prediction. Feedback outcomes are matched to the relevant query using the platform, application session, and shipment identifier.

2. Subsequent query behavior. At the platform-device-shipment level, we measure whether the shipment is queried again within one hour, six hours, and twenty-four hours after its first experimental query; the number of additional queries within twenty-four hours; the logarithm of one plus the number of additional queries; and the elapsed time until the next query.

3. Precision choice in Phase I. Among users who interact with the precision-selection interface, we measure whether the final precision differs from the randomized default, whether the user chooses a finer or coarser precision, the final selected precision, and the time spent making the selection where click-level timing is available.

The primary causal analysis of fixed display precision uses Phase II, in which users could not change the randomized precision. Phase I is used to estimate the effect of randomized default precision and to study active precision choice.

This study was a large-scale randomized field experiment embedded in the shipment-tracking interface of a digital logistics platform. The randomization unit was a platform-device-shipment combination. When a device first queried a particular shipment during the experiment, the platform’s backend system used a computer-generated pseudo-random procedure to assign that combination to an experimental condition. Later queries of the same shipment by the same device retained the initial assignment.

The experiment used a 21-arm factorial structure. Twenty prediction arms varied three dimensions: prediction technology, display precision, and, where applicable, route-information disclosure. The prediction technologies were a traditional model, an AI maximum-probability rule, and an AI fastest-time rule. Display precision was randomized among one day, six hours, one hour, and thirty minutes. Within the two AI conditions, route disclosure was additionally randomized between displaying only the final delivery-time prediction and displaying the final prediction together with predicted intermediate logistics nodes. Because the traditional model did not generate intermediate-node forecasts, route disclosure was not randomized in that condition. A separate twenty-first arm displayed no prediction.

Assignment was hierarchical rather than uniform across the 21 final arms. The backend first assigned the broad prediction condition or technology, then randomized route disclosure within the AI conditions, and finally randomized display precision within each applicable model-by-display cell.

The experiment was conducted in two phases. Phase I ran from May 3 through May 12, 2026. Random assignment determined the default display precision, but users could change it before confirmation. Phase II ran from May 13 through June 1, 2026. Users could not change the randomly assigned precision. Thus, Phase I identifies the effect of a randomized default and provides information about endogenous precision choice, while Phase II identifies the effect of fixed display precision.

The intervention changed only the prediction and information displayed in the tracking interface; it did not affect shipment handling or delivery. User queries and feedback were recorded during the experimental period. Shipment-trajectory records were retained through June 10, 2026, to recover realized node and delivery times for shipments that had not yet arrived when the intervention ended.

Not available

Randomization was conducted automatically by the platform’s backend computer system when an eligible platform-device-shipment combination first entered the experiment. A computer-generated pseudo-random algorithm assigned the prediction condition and, conditional on that assignment, the applicable prediction technology, route-disclosure condition, and display precision. The procedure was hierarchical and therefore did not assign equal probabilities to all 21 final arms. Assignment was implemented without manual intervention and recorded in the platform’s experimental logs.

The randomization unit was a platform-device-shipment combination, identified by platform, device identifier, and shipment waybill. Treatment was assigned at the first experimental query for that combination and retained for its subsequent queries. Different shipments queried by the same device constituted separate randomization units and could receive different assignments.

All treatment dimensions were randomized at this same level. There was no higher-level randomization by user account, city, courier company, route, or other cluster.

No

The experiment was not cluster-randomized. Approximately 1,000,000 platform-device-shipment combinations were planned as randomization units.

Approximately 1,400,000 shipment-tracking query-session records during the experimental period, while multiple query-session records could be observed for the same platform-device-shipment randomization unit.

Assignment was hierarchical, so the 21 final arms were not designed to have equal sample sizes. Of approximately 1,000,000 planned randomization units, the four broad conditions were expected to receive approximately 250,000 units each:
No prediction: approximately 250,000 units in one arm.

Traditional prediction model: approximately 250,000 units, divided equally across four precision arms, or approximately 62,500 units per arm.

AI maximum-probability rule: approximately 250,000 units, divided equally across two route-disclosure conditions and four precision levels, or approximately 31,250 units per arm across eight arms.

AI fastest-time rule: approximately 250,000 units, divided equally across two route-disclosure conditions and four precision levels, or approximately 31,250 units per arm across eight arms.

Actual realized sample sizes could differ because of platform traffic, logging availability, and the timing of eligible shipment queries.