Effect of the product order and the product review on the product choice.

Last registered on March 21, 2023


Trial Information

General Information

Effect of the product order and the product review on the product choice.
Initial registration date
March 19, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
March 21, 2023, 4:47 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.



Primary Investigator

Tokyo Keizai University

Other Primary Investigator(s)

PI Affiliation
Kyoto University
PI Affiliation
Kyoto University
PI Affiliation
Keio University

Additional Trial Information

In development
Start date
End date
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
To investigate the impact of product reviews and product orders on consumer behavior in online retail, we will recruit 5,000 participants for the experimental websites designed to simulate an online retail store. We will assign several treatments to investigate the effect of information on consumer behavior, including two types of user reviews, three types of product orders, three types of review orders, two-sided information provisions, and a warning message against the existence of incorrect or misleading reviews.

The first type of review is based on reviews that have been deleted from an online retail website, while the other type of review is based on reviews that have not been deleted. We will sort the order of products by user rating for products, by the number of user ratings for products, and by random. We will sort the order of reviews by user rating for reviews, by newest, and by random.

To determine the willingness to pay for a product, we will use an incentive-compatible questionnaire.

Additionally, we will collect information from participants regarding their experience with online retailing, their level of trust in online retailing, risk attitudes, information-seeking attitudes, participating environments, impressions of our experimental websites, and socio-demographic characteristics. We will also record click log data from our experimental websites. Using this information, we will estimate the treatment effect of information structure on consumer behavior, the effect of the information on the consumer surplus, search cost, information processing costs, and the prior for the goods to better understand the treatment effect.
External Link(s)

Registration Citation

Ida, Takanori et al. 2023. "Effect of the product order and the product review on the product choice.." AEA RCT Registry. March 21. https://doi.org/10.1257/rct.11020-1.0
Experimental Details


Participants will be asked to access the websites that the investigators have designed. The first step is to answer some questions about their experience with online retailing. Next, participants will be asked to choose their preferred products from three different product categories on the website that simulate an online retail store. The website will provide natural language user reviews, the number of reviews, user ratings for products, and specifications for the products. The reviews, number of reviews, and ratings will vary among participants. In addition to the provided information, we will also assign three different orders for the products listed on the product list page and three different orders for the reviews on the product detail pages. Additionally, we will randomly assign product detail pages with two-sided information provision that shows the pros and cons of the product. Furthermore, we will randomly assign notifications on the existence of fake or infrared reviews.
Intervention Start Date
Intervention End Date

Primary Outcomes

Primary Outcomes (end points)
Changes in the information provided and the order in which it is presented can impact consumer behavior, including product choice, information-seeking behavior, and willingness to pay for products.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
We will investigate the heterogeneity of treatment effects between the experience of using EC and interest in the product, which has been shown in previous studies. In addition, using click-log data, we can estimate the search cost and information processing costs of consumers to investigate consumer behavior in counterfactual situations.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The experiments will be conducted in March 2023. We will recruit 5,000 adult participants which represent internet users in Japan. To recruit participants, we ask an internet survey firm to send an email that asks about their willingness to participate in the online survey for online shopping with the survey periods and the fact that the survey will be conducted by academic researchers. The e-mail also informs the candidate respondents that they will be awarded 250JPY equivalent points and that a small number of them will be given the product they choose in the survey or additional points. Then, the survey firm will send e-mails with the URL of experimental websites to people who accept participating in the survey. Emails containing the URL of the experimental website are sent by the firm to candidate respondents until the obtained response can reach the target number which represents Japanese Internet users by gender and age.

The website first asks the participants about their experience in online retail. Then, the website asks respondents to choose the most wanted products or none of the products are wanted for a category of products in pages that simulate an online retail store. We pick digital cameras, headphones, and smartwatches as categories of products of store pages. We pick 10 digital cameras, 30 headphones, and 30 smartwatches for each product list, respectively. The order of categories is randomized.

The simulated store pages consist of three levels: a product listing page, a product details page, and a page displaying all reviews.

On the first level page, we show a list of products in one category. In the product list, we show the product's name, the average user rating score, the number of reviews, and the picture of the product. When respondents click on the "see details of a product" button, the website shows the 2nd level page of the product. Once the respondents proceed to the 2nd level page and return to this page, the first level page adds the "None of them are wanted" button at the top of the product list. The "None of them are wanted" button leads respondents to the "confirmation page".

On the second level page, the detail of the product contains the specifications of the product and five reviews for the product with ratings for the review. The product detail page has three buttons. The first one is the "return to the list" button, which proceeds the respondents to the first level page. The second is the "choose it" button, which proceeds respondents to the confirmation pages. The third is the "None of the products are wanted" button. The fourth is the "review is helpful" button. The fifth is "see more reviews", which proceeds respondents to the third level page. The product detail pages also may show two-sided presentations by respondents' assigned treatment.

On the third level page, we show 10 reviews and a two-sided presentation. The third level page has two buttons. The first is the "return to the page before", which proceeds respondents to the 2nd level pages. The second is the "review is helpful" button.

When the respondents proceed to the confirmation page, we show the "confirm" button and the "back to the product page" button or "back to the product list page" depending on the previous page. Once respondents push the "confirm" button, the website does not allow respondents to back to previous pages.

We randomize participants on the two sources of information, three orders of products, three orders of reviews, whether the product page contains a two-sided presentation of reviews or not, and whether the product page contains a warning message against the existence of
incorrect or misleading reviews.

After the three product categories choice experiments, we ask the respondents about their impressions and their experience of the websites, along with their experience and perceptions concerning their use of online shopping and the Internet.
Experimental Design Details
After respondents' confirmation of the product choice, the website asks respondents whether they would like to participate in a draw to receive a product or a certain amount of money. We also inform them about the account of money is randomly assigned and the opportunity of choice is once. This design enables us to estimate the willingness to pay (WTP) for the product in an incentive-compatible manner.

Other than those randomizations mentioned in public, we also randomize the order of product categories and the amount of money for WTP questionary.

The survey contains two traps. The first trap confirms whether the respondents read a text that explains the experiments. The text asks respondents to select 'None of the options apply' and shows 11 options. The first trap is inserted between the second categories of product choice and the third categories of product choice. The second trap confirms whether the respondents ignored the price of products in the choice experiments by asking what product attributes were important in the selection of products on this website. We show a list of attributes to choose which contains "price" and other attributes which are frequently described on product detail pages after three categories of product choice are made.
Randomization Method
Randomizations are done in the website server using SHA1PRNG, a pseudorandom number generation (PRNG) algorithm. For randomization unit 1) described below, adaptive randomization (biased coin design) is also adopted based on Rosenberger, W. F. (2002).
Randomization Unit
Four randomizations are implemented as follows:
1) Randomization to allocate participants to 36 groups for combinations of all treatments except for a warning message.
2) Randomization to allocate participants to four groups to present different amounts of money in the WTP question.
3) Randomization to allocate participants to two groups to determine whether a warning message is to be displayed or not.
4) Randomization to allocate participants to six groups to determine orders of the three product categories to be shown on the website.
For each participant, the first two randomizations are performed by each product category.
This means that these two randomizations are carried out three times for a single participant.
On the other hand, the last two randomizations, which determine the warning message that is expected to be effective across the categories once displayed, and also determine the order of the product categories, are performed only once for a single participant.
Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters
Sample size: planned number of observations
Sample size (or number of clusters) by treatment arms
We will assign the following treatments.
1. two sources of information (deleted: not deleted(control))
2. three orders of products (random(control):ratings of product:number of reviews)
3. three orders of review (random(control):newest:ratings of review)
4. two types of information provisions (with notification about fake or infrared review: without it)
5. two types of products review (with two-sided presentations or without them)
Therefore, the number of combinations of treatments is 72 for each product category.
Therefore, we randomly assign each combination of treatments to 69 or 70 respondents at random.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We set the significant level as 0.05 and power as 0.8. Then, when the treatment is binary, we will find a mean difference about 0.079 and when the treatment is trinary, we will find a mean difference about 0.097.

Institutional Review Boards (IRBs)

IRB Name
Global Survivability Studies Unit, Center for the Promotion of Interdisciplinary Education and Research, Kyoto University
IRB Approval Date
IRB Approval Number
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information


Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information


Is the intervention completed?
Data Collection Complete
Data Publication

Data Publication

Is public data available?

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials