|
Field
Last Published
|
Before
March 20, 2025 03:43 AM
|
After
March 20, 2025 09:12 PM
|
|
Field
Intervention (Public)
|
Before
The intervention involves displaying expert reviews on a web-based medication review platform exclusively for licensed physicians. On this platform, physicians evaluate medications and provide written reviews. Expert reviewers are categorized into three types: Top Reviewers (based on "likes" received from peer physicians), Veteran Reviewers (based on prescription volume), and Authority Reviewers (recognized Key Opinion Leaders). When physicians evaluate medications on the platform, they are randomly assigned to one of four groups:
- Treatment Group A: Shown reviews from Top Reviewers
- Treatment Group B: Shown reviews from Veteran Reviewers
- Treatment Group C: Shown reviews from Authority Reviewers
- Control Group: No expert reviews shown
Each expert review includes both a numerical rating (5-point scale for overall satisfaction) and written comments about the medication.
|
After
The intervention is conducted on a secure web-based medication review platform that is accessible exclusively to licensed physicians. On this platform, physicians evaluate medications by submitting a numerical rating (on a five-point overall satisfaction scale) and providing detailed written comments based on their clinical experiences.
To examine how expert reviews influence physicians’ evaluations, participants are randomly assigned at the review level to one of five experimental groups:
Treatment Groups
Group A: Shown evaluations from Veteran Reviewers
Group B: Shown evaluations from Top Reviewers
Group C: Shown evaluations from Authority Reviewers
Group D: Shown evaluations with reviewer identity anonymized
Control Group
Group O: Shown no expert evaluations
Each expert evaluation includes both a numerical satisfaction rating and a written comment describing the expert's experience with the medication.
|
|
Field
Experimental Design (Public)
|
Before
The study employs a randomized controlled trial design with three treatment arms and one control arm. The study participants are licensed physicians who are registered on the web-based review board and actively prescribe medications in the therapeutic areas under study. Participating physicians are randomly assigned based on their member ID to receive one of the three types of expert reviews or no expert review (control) when evaluating medications. The study focuses on medications commonly used for treating major chronic diseases and psychiatric disorders.
|
After
The study employs a randomized controlled trial design with four treatment arms and one control arm. The study participants are licensed physicians registered on the web-based review board who actively prescribe medications in the relevant therapeutic areas. Participating physicians are randomly assigned based on their member ID to one of the three expert review types, a review without reviewer type information (treatment), or no expert review (control) when evaluating medications. The study focuses on medications commonly used for treating major chronic diseases and psychiatric disorders.
|
|
Field
Randomization Method
|
Before
Randomization is conducted by computer at the time when physicians submit medication reviews on the web-based review board. Using a random number generator, each reviewing physician is randomly assigned to one of the four experimental groups (three treatment groups or control) before they begin their medication evaluation. The randomization is conducted at the physician level to ensure that the same physician consistently receives or does not receive expert reviews throughout the study period, preventing potential contamination between treatment and control conditions.
|
After
Randomization is conducted by computer at the time physicians submit medication reviews on the web-based review board. Using a random number generator, each reviewing physician is randomly assigned to one of the five experimental groups (four treatment groups or control) before they begin their medication evaluation. The randomization is conducted at the physician level to ensure that each physician consistently receives or does not receive expert reviews throughout the study period, preventing potential contamination between treatment and control conditions.
|
|
Field
Randomization Unit
|
Before
The unit of randomization is the reviewing physician. When a physician initiates a medication review on the platform, they are randomly assigned to either one of the three treatment groups (shown an expert review) or the control group (no expert review shown).
|
After
The unit of randomization is the reviewing physician. When a physician initiates a medication review on the platform, they are randomly assigned to either one of the four treatment groups (shown a review) or the control group (no review shown).
|
|
Field
Planned Number of Observations
|
Before
For each medication analyzed, the target minimum sample size is 216 reviews (54 reviews × 4 groups) if control variables are included, or 108 reviews (27 reviews × 4 groups) if control variables are not included.
|
After
For each medication analyzed, the target minimum sample size is 270 reviews (54 reviews × 5 groups) if control variables are included, or 135 reviews (27 reviews × 5 groups) if control variables are not included.
However, these sample sizes apply to the full experimental scheme, which includes four treatment groups (A, B, C, D) and one control group. If only the partial experimental scheme (with Treatment Groups A, D, and the Control Group) is conducted, the total sample size will be smaller. Nevertheless, we aim to secure at least 54 reviews per group if control variables are included, or 27 reviews per group if control variables are not included.
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
The study will collect data from medication reviews across multiple therapeutic areas. For each medication analyzed, we target at least 54 reviews per group (treatment groups A, B, C, and control group O) to achieve 80% power at a 0.05 significance level for detecting a medium effect size. In this analysis, we perform three separate regression analyses, each comparing one treatment group (A, B, or C) with the control group (O).
For each medication analyzed:
-Treatment Group A (Top Reviewer): 54 reviews
- Treatment Group B (Veteran Reviewer): 54 reviews
- Treatment Group C (Authority Reviewer): 54 reviews
- Control Group O: 54 reviews
This setup allows for sufficient statistical power across three distinct comparisons: (A vs. O), (B vs. O), and (C vs. O).
|
After
The study will collect data from medication reviews across multiple therapeutic areas. For each medication analyzed, we target at least 54 reviews per group (treatment groups A, B, C, D and control group O) to achieve 80% power at a 0.05 significance level for detecting a medium effect size. In this analysis, we perform three separate regression analyses, each comparing one treatment group (A, B, or C) with the control group (O).
For each medication analyzed:
- Treatment Group A (Veteran Reviewer): 54 reviews
- Treatment Group B (Top Reviewer): 54 reviews
- Treatment Group C (Authority Reviewer): 54 reviews
- Treatment Group D (Non-specified Reviewer): 54 reviews
- Control Group O: 54 reviews
This setup allows for sufficient statistical power across three distinct comparisons: (A vs. O), (B vs. O), (C vs. O), and (D v.s O).
For each medication analyzed:
-Treatment Group A (Top Reviewer): 54 reviews
- Treatment Group B (Veteran Reviewer): 54 reviews
- Treatment Group C (Authority Reviewer): 54 reviews
- Control Group O: 54 reviews
This setup allows for sufficient statistical power across three distinct comparisons: (A vs. O), (B vs. O), and (C vs. O).
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
For each medication analyzed, we aim to detect a medium effect size with a 0.05 significance level and 80% power, focusing on two main outcomes: physician rating scores and comment content similarity.
(1) For rating scores on a 5-point scale, we can detect a minimum effect size of 0.15 (Cohen's f2), equivalent to a 0.3-point difference. When including control variables (e.g., gender, age, employment type), a sample size of 108 reviews (divided evenly between treatment and control) is required. Without control variables, 54 reviews (also divided evenly between treatment and control) are sufficient. The regression model will estimate the influence of expert ratings on general physicians' scores, adjusting for demographic factors.
(2) For comment similarity, we can detect a minimum effect size of 0.25 (Cohen's f), requiring 248 pairwise comparisons per group. This translates to a minimum of 16 comments per group, or 64 comments in total, to achieve sufficient pairwise comparisons. An ANCOVA analysis will compare group differences in similarity, assessing whether exposure to different expert types influences the language used in physician reviews.
- For further details, refer to the Pre-Analysis Plan document.
|
After
For each medication analyzed, we aim to detect a medium effect size with a 0.05 significance level and 80% power, focusing on two main outcomes: physician rating scores and comment content similarity. (1) For rating scores on a 5-point scale, we can detect a minimum effect size of 0.15 (Cohen's f2), equivalent to a 0.3-point difference. When including control variables (e.g., gender, age, employment type), a sample size of 108 reviews (divided evenly between treatment and control) is required. Without control variables, 54 reviews (also divided evenly between treatment and control) are sufficient. The regression model will estimate the influence of expert ratings on general physicians' scores, adjusting for demographic factors. (2) For comment similarity, we can detect a minimum effect size of 0.25 (Cohen's f), requiring 270 pairwise comparisons per group. This translates to a minimum of 16 comments per group, or 80 comments in total, to achieve sufficient pairwise comparisons. An ANCOVA analysis will compare group differences in similarity, assessing whether exposure to different expert types influences the language used in physician reviews. For further details, refer to the Pre-Analysis Plan document.
|
|
Field
Intervention (Hidden)
|
Before
The intervention is implemented on a web-based medication review board where licensed physicians share their experiences with medications through ratings and detailed comments. Expert reviews are displayed immediately before a physician begins their medication evaluation. Expert reviewers are defined using strict criteria:
- Top Reviewers: Physicians in the top 10% by total "likes" received from peer physicians
- Veteran Reviewers: Physicians in the top 10% by prescription volume and patient count
- Authority Reviewers: Recognized Key Opinion Leaders (KOLs) or Area Opinion Leaders (AOLs) in the relevant therapeutic area
|
After
The intervention is implemented on a web-based medication review platform where licensed physicians routinely share their experiences with various drugs through structured ratings and written narratives. In the experiment, expert evaluations are displayed immediately before the participant begins their own assessment. Expert reviewers are identified using objective and pre-specified criteria as follows:
- Top Reviewers: Physicians ranked in the top 10% based on total “likes” received from other physicians on the platform
- Veteran Reviewers: Physicians ranked in the top 10% by both prescription volume and patient count
- Authority Reviewers: Recognized clinical leaders such as Key Opinion Leaders (KOLs) or Area Opinion Leaders (AOLs) in the relevant therapeutic domain
The full experimental design includes all five arms described in the public protocol (Groups A through D, and Group O). However, due to practical considerations such as time constraints or budget limitations, a partial implementation may be adopted. The partial design comprises only three groups: Group A (Veteran Reviewers), Group D (Anonymous Reviewer Information), and Group O (Control).
The initial experiment timeline was set from November 11, 2024 to June 30, 2025. However, in coordination with our partner MedPeer Inc., the launch was rescheduled to April 11, 2025 to accommodate refinements in the experimental procedure. Prior to the full-scale implementation, a pilot study involving a single medication will be conducted between March 12 and March 20, 2025.
|