BallotBot: Can Chatbots Strengthen Direct Democracy?

We randomly assign a random sample of Swiss voters to use a chatbot or a voter guide to answer a set of closed-ended questions about the issues they vote for on the 9th of June. We measure the time they need to complete the task, the perceived cost, the share of correct answers, how sure they are of each answer, and their willingness to ask further questions. We do this to understand why these two sources of information are different and which one works better in the context of mapping political information into knowledge. Ultimately, we aim to shed light on the potential beneficial role of AI in strengthening direct democracy.

2024-05-17

2024-06-09

Share of correct answers, degree of confidence, objective cost of answering the questions, perceived cost of answering, willingness to ask further questions, and time spent on the resource for answering further questions.

The degree of confidence is measured on a 4-degree scale (how certain are you about your answer? 1.highly uncertain 2.uncertain 3.certain 4.highly certain). The objective cost is measured as the time they need to answer each question. The perceived cost is measured through a bidding game similar to the Becker–DeGroot–Marschak method. The willingness to ask further questions is measured through the probability of choosing a question a participant is interested in and looking for the answer at the end of the survey, without any incentive in place. The time spent on the resource is measured as the amount of seconds the respondent spends outside the survey window.

We collect some feedback on the functioning of the chatbot, from the experimental group that received it to explore how they compare it with the voter guide.

It is a survey experiment. After collecting the baseline characteristics of the participants (demographics, socioeconomic conditions, attitudes toward AI, baseline knowledge of the election matters), we randomize the respondents into two groups. The first group is given the PDF version of the official voter guide (a document that the government distributes to all the voters), and the second group is given an internally developed chatbot trained on the voter guide's text to answer any query about election matters. Respondents answer 5 closed-answer questions concerning election matters using the resource they are randomized into. For each correct answer, they earn 0.2 CHF on top of their participation fee. Then, they can answer an additional question and bid for their remuneration. A random number is drawn. If the number is higher than the bid, the respondent does not take the additional question, otherwise, she takes it and gets paid her bid in case of correct answer. Then, respondents are encouraged to ask a question of their choice to the resource. This is not incentivized. Finally, we collect feedback on the functioning of the chatbot to the chatbot group.

Qualtrics randomization mechanism.

Individual.

No

/

1000 individuals

/