The Effect of Formal-relational Contract Management Training in the Public Sector

Last registered on September 20, 2023

Pre-Trial

Trial Information

General Information

Title
The Effect of Formal-relational Contract Management Training in the Public Sector
RCT ID
AEARCTR-0012059
Initial registration date
September 16, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 20, 2023, 10:54 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Roskilde University

Other Primary Investigator(s)

PI Affiliation
Roskilde University
PI Affiliation
University of California - Santa Barbara

Additional Trial Information

Status
In development
Start date
2023-09-17
End date
2025-06-30
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This trial examines the effect of formal-relational contract management training in a national- scale randomized field experiment in Denmark. Participants are local and regional public contract managers receiving a 5 ECTS (European Credit Transfer System) masterclass training in formal-relational contract management.
External Link(s)

Registration Citation

Citation
Brogaard, Lena, Ole Helby Petersen and Matthew Potoski. 2023. "The Effect of Formal-relational Contract Management Training in the Public Sector." AEA RCT Registry. September 20. https://doi.org/10.1257/rct.12059-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The study is based on a focused 5 ECTS (European Credit Transfer System) master level class in formal-relational contract management as a free course offered to all local and regional government contract managers (purchasing officers, contract managers, heads of contracting offices, and similar job categories) in Denmark. The trial is a delayed- start treatment design with two interventions. Contract managers assigned to the treatment group receive the training in the first wave (September to November 2023), and those the control group receives the training in the second wave (September to November 2024).
Intervention Start Date
2023-09-17
Intervention End Date
2024-12-31

Primary Outcomes

Primary Outcomes (end points)
The trial focuses on three groups of outcomes:
1) Program Learning. Did the program increased participants’ ability to and confidence (self-efficacy) in using a FRC approach to prepare and manage contracts?
1a: Public contract managers' ability to use FRC when preparing and managing contracts.
1b: Public contract managers' self-efficacy in using FRC when preparing and managing contracts.
2) Contract Management Practice. Did the program increase participants’ use of FRC writing and enforcement practices?
2a: Public contract managers' use FRC shared goals and guiding principles.
2b: Public contract managers' use of four FRC governance mechanisms: 1) penalties and rewards, 2) communication and information sharing, 3) dispute resolution, 4) procedures for contract amendments.
2c: Public contract managers’ use of FRC enforcement to manage the buyer-seller relation during the contract period –the four FRC governance mechanisms mentioned in 2b.
3) Contract Performance. Did the program improve the outcomes of contracts for the buyers and sellers?
3a: Buyers' and sellers' perception of value from the purchase (price, product quality, relationship satisfaction).
3b: Buyers’ and sellers’ perceptions of whether the other party was behaving in a “cooperative” spirit in line with FRC objectives.
3c: Buyer’s perception of transaction costs (time, effort and other resources spent to make the transaction happen).

We will correct for multiple testing within each of these three groups of outcomes.
Primary Outcomes (explanation)
Outcome 1a will be measured through coding of a 2-page contracting case assignment completed by contract managers before the training starts and revised and re-submitted after the training. The assignment asks contract managers to respond to questions about a contracting case they are working on in their job. Outcome 1a measures the extent to which contract managers use two specific FRC practices when preparing and managing their contracts: 1) shared goals and guiding principles for the buyer and seller, and 2) four FRC governance mechanisms: 1) penalties and rewards, 2) communication and information sharing, 3) dispute resolution, 4) procedures for contract amendments. The case assignment is provided as course material prior to the start of training and revised and resubmitted after the training.

The case assignment will be coded by the researchers, with assistance from research assistants, for the presence of the two FRC practices: 1) shared goals and guiding principles for the buyer and seller, and 2) the four FRC governance mechanisms as described above, with the coders being blind to whether the cases are from participants in the control group or the treatment group and whether each case was completed before or after the training. Cases will be blinded and randomized in order when coded. The researchers may experiment with Natural Language Processing/AI procedures for coding. In that case, the researchers will train a language model and provide a test of its validity compared to manual coding done by the researchers – a procedure and test that will be reported in published papers.

Outcome 1b will be measured via a participant survey distributed before and after the training. The survey instrument is provided as supplementary material in the original language of the contract managers to document the precise question formulation. The survey questions are attached in a supplementary file. The survey will be translated to English in appendices to published papers.

Outcome 2 will be measured through coding of officially tendered contracts written by participants in the treatment and control group before and after the course. We have built robot process automatization programs (RPAs) to download all contract data and documents (such as requests for proposals, product descriptions, and contracts) for all contracts tendered by local and regional governments in Denmark from 1 January 2021 onwards. We run the RPA processes every week and manually inspect and quality assure the downloads for accuracy and completeness by the researchers and a team of research assistants (also done weekly). We code the contract documents for two types of formal-relational contracting elements:

2a) Shared goals and guiding principles

2b) Four FRC governance mechanisms: 1) penalties and rewards, 2) communication and information sharing, 3) dispute resolution, 4) procedures for contract amendments. The researchers, with assistance from research assistants, will code the RPA downloads for the presence of these governance mechanisms with the coders being blind to whether the RPAs are from participants in the control group or the treatment group, and with RPA downloaded contracts appearing in randomized order. The researchers may experiment with Natural Language Processing/AI procedures for coding to increase speed and accuracy. In that case, the researchers will train a language model and provide a test of its validity compared to manual coding done by the researchers – a test that will be reported in published papers.

2c) Public contract managers use FRC enforcement mechanisms to manage the buyer-seller relation during the contract period – the four FRC governance mechanisms mentioned in 2b) – measured in a survey of public contract managers with survey items for each of the four FRC governance mechanisms.

Conceptual definition of outcomes: The coding of shared goals and guiding principles will be based on the definitions offered in conceptual work on FRC by Frydlinger, Hart, and Vitasek (2019) and Frydlinger et al. (2021). The coding of the four FRC governance mechanisms will build on the same works and measure whether the core principles of FRC are reflected in how public contract manage these governance mechanisms.

References:
Frydlinger, D., Hart, O., & Vitasek, K. (2019). A new approach to contracts: how to build better long-term strategic partnerships. Harvard Business Review, 97(5), 116-126.

Frydlinger, D., Vitasek, K., Bergman, J., & Cummins, T. (2021). Contracting in the new economy: using relational contracts to boost trust and collaboration in strategic business relationships. Palgrave Macmillan.

Outcome 3a-3c will be measured by surveys of buyers and sellers a year after the training is completed. The survey instrument measures buyers' and sellers' satisfaction with the contract relationship, buyers’ perception of value from the purchase, buyers’ and sellers’ perceptions of whether the other party was behaving in a “cooperative” spirit in line with FRC objectives, and buyers’ and sellers’ perception of how much buyer’s transaction costs (time, effort and other resources spent to make the transaction happen) in phases of the contracting process: market search, contract writing, negotiation/contract award, contract signature, monitoring, enforcement.

Secondary Outcomes

Secondary Outcomes (end points)
After completion of the treatment, we will perform exploratory analyses of additional connections between the treatment and possible outcomes within different subgroups such as simple versus complex products and between different types of contract managers (experience, education, gender) and characteristics of buying agencies. The aim of the secondary outcomes is to identify potential areas for additional research.

To explore whether the effects of treatment differ across different products, we will conduct secondary analyses of outcomes 1, 2 and 3 at varying levels of product complexity. In addition, we will conduct exploratory analyses using additional data culled from the RPA contracts.
Secondary Outcomes (explanation)
The secondary outcomes will be evaluated by examining primary outcomes at differing levels of product complexity. Product complexity was measured by the project researchers in a survey for the 60 most common product categories and by using AI and natural language processing on contract data for all other product categories. These measures will be made publicly available in working papers before being used for secondary outcome analyses and will thus be results-blind.

The secondary outcomes for contract manager experience, education, and gender is sourced from signup information provided by contract managers when enrolling for the FRC master level course, and in the survey (current job category). Exploratory analyses with characteristics of buying agencies draw on administrative register data.

Contract performance measurements based on data culled from RPA downloads will focus on contract performance indicators. Additional contract outcomes from RPA contracts: early contract termination; evaluation criteria; and the level of detail in product specification, formal governance mechanisms, and exchange terms.

Experimental Design

Experimental Design
The participants consist of all local and regional government public contract managers in Denmark who volunteered for the contract management training. The participants are stratified by organization type and size, and randomly assigned to the treatment group or the control group, with the treatment group receiving treatment in Fall 2023 and the control group receiving a delayed-start treatment in the Fall of 2024. The contract management training consists of a focused 5 ECTS (European Credit Transfer System) course in formal-relational contracting in the public sector, developed by the research team for this trial.
Experimental Design Details
Not available
Randomization Method
In office by a computer
Randomization Unit
This is a cluster randomized trial. Public contract managers are clustered at their organizations (municipalities, regions, local public utility companies, local public transport companies).
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
82 local and regional purchasing agencies
Sample size: planned number of observations
208 public contract managers
Sample size (or number of clusters) by treatment arms
App. 41 purchasing agencies receive contract management training in the first wave (September to November 2023). App. 41 purchasing agencies (control group) receive contract management training in the second wave (September to November 2024).
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number