Randomization method. For legal and logistical reasons, the Brazilian Court of Audits (TCU) randomizes the treatment allocation and sends the messages to our receiving organization (DNIT). The randomization process happens in four steps. First, TCU's artificial intelligence tools select each purchase act with at least one early-warning alarm. Second, the software Microsoft Power BI generates a data entry identified by the pair (purchase act ID, type of alarm), uniquely identifying each group of early earnings alarms within each purchase acts. Third, the software allocates each entry to control and treatment groups and prepares an automated e-mail for each distinct alarm in the data entries assigned to the treatment group. Fourth, for each purchase act, the chief of DNIT's internal control receives the automated e-mails and forwards them to the local office manager implementing the purchase act.
We describe a didactical example to increase our readers' understanding. Assume that the purchase act with identifier PA1 has five early warning alarms belonging to two groups: two text patterns suggesting restriction to competition (RC) and three indicating opportunities for efficiency gains (EG). In this case, the software generates two data entries - i.e., PA1-RC and PA1-EG - and two randomizations. If we assume that the software randomly allocates PA1-RC to the treatment group and PA1-EG to the control group, the manager of the purchase act receives two e-mails, one for each early warning alarm suggesting restriction to competition (RC).