Robo-Advising Meets Large Language Models: Educating Investors on Alpha and Beta of Mutual Funds and Stocks

To investigate the impact of financial literacy on investor behavior and performance, we are conducting a Randomized Controlled Trial (RCT) involving 28,819 investors who expressed interest in participating. We randomly selected 14,000 of these investors to form the treatment group, while the remaining 14,819 investors constitute the control group. Both groups are similar in terms of investment portfolio size and personal characteristics, ensuring a balanced and fair comparison.

In the treatment group, investors receive targeted financial education on the concepts of beta and alpha through our robo-advising platform. The education module provides detailed explanations, examples, and personalized insights into how these metrics influence investment risk and performance. Investors in the control group continue to receive standard robo-advisory services without the additional financial literacy component. Moreover, a subsample of the treatment group is further enhanced with a back-end Large Language Model (LLM) system or other follow-up assistance for interactive learning.

The robo-advisor begins by evaluating the investor’s current portfolio holdings, systematically assessing each mutual fund or stock by calculating key metrics such as alpha and beta. It performs a multi-factor analysis using a comprehensive set of factors, including size (SMB), value (HML), and momentum (MOM), among others. Statistical measures, such as t-statistics, are computed to determine the significance of alpha, distinguishing genuine manager skill from random fluctuations. The robo-advisor aggregates these metrics to provide an overall assessment of the portfolio, benchmarking against market indices to provide context on the portfolio’s performance and volatility.

Based on this detailed analysis, the robo-advisor generates personalized recommendations for the investor. These recommendations aim to optimize the portfolio by suggesting actions such as reallocating funds from underperforming assets, maintaining positions in high-performing assets, and introducing low-cost, diversified investment vehicles like ETFs or index funds. The ultimate goal is to help the investor achieve better returns, manage risk more effectively, and minimize investment costs.

2023-11-01

2027-04-01

1. Portfolio Composition:
o Changes in the allocation of assets within investors' portfolios, specifically the proportion of investments in ETFs, index funds, active mutual funds, and individual stocks.
2. Trading Activity:
o The frequency and volume of trades executed by investors, including the number of buy and sell transactions.
3. Investment Performance:
o Monthly returns on investment portfolios, measured in basis points.
o Average monthly gains, specifically tracked in terms of RMB for this study.
4. Risk Management:
o Changes in portfolio risk metrics, such as beta and the standard deviation of returns.
o The overall risk-adjusted performance, including metrics like Sharpe ratio and alpha.
5. Financial Literacy Improvement:
o Assessment of investors' understanding of key financial concepts (beta, alpha) through pre- and post-education surveys.
o The ability of investors to make informed decisions based on these concepts.
6. Behavioral Changes:
o Adoption of recommended investment strategies, including the extent to which investors follow personalized recommendations.
o The impact of interactive learning tools (LLM chatbot, human advisors, etc.) on the likelihood of implementing these strategies.
7. International Diversification:
o The extent to which investors diversify their portfolios internationally, even when not initially recommended by the robo-advisors.

Experimental Design (Public) This study is a Randomized Controlled Trial (RCT) conducted in collaboration with one of the largest brokerage firms in China. The objective is to investigate the impact of financial literacy on investor behavior and performance. The trial involves 28,819 investors who expressed interest in participating. The experimental design is as follows:

Participant Allocation: We randomly selected 14,000 of these investors to form the treatment group, while the remaining 14,819 investors constitute the control group. The randomization was performed using a computer algorithm to ensure transparency and reproducibility.

Interventions: Treatment Group: Investors in the treatment group receive targeted financial education on the concepts of beta and alpha through our robo-advising platform. The education module includes detailed explanations, examples, and personalized insights into how these metrics influence investment risk and performance. This group is further divided into four subgroups to examine the heterogeneity of the intervention’s effects:

ChatGPT/DeepSeek (LLM) Chatbot: Receives financial education and personalized advice through an LLM-powered chatbot.
Without Chatbot: Receives financial education through static content such as articles and videos, without interactive chatbot support.
Human Chat Assistant: Has access to human financial advisors who provide real-time assistance and explanations via chat.
Normal Chatbot: Uses a standard rule-based chatbot that provides predefined responses and guidance based on a fixed set of rules.
Additionally, each subgroup is divided into two groups:

Automatic Execution: This group benefits from automatic execution or one-click execution of the recommended portfolio.
DIY (Do It Yourself): This group can only execute the recommended portfolio by doing it themselves.
Control Group: Investors in the control group continue to receive standard robo-advisory services without the additional financial literacy component.

Assessment and Monitoring: The robo-advisor begins by evaluating the investor’s current portfolio holdings, systematically assessing each mutual fund or stock by calculating key metrics such as alpha and beta. It performs a multi-factor analysis using a comprehensive set of factors, including size (SMB), value (HML), and momentum (MOM), among others. Statistical measures, such as t-statistics, are computed to determine the significance of alpha.

Based on this detailed analysis, the robo-advisor generates personalized recommendations for the investor. These recommendations aim to optimize the portfolio by suggesting actions such as reallocating funds from underperforming assets, maintaining positions in high-performing assets, and introducing low-cost, diversified investment vehicles like ETFs or index funds.

Outcome Measurement: Throughout the study period, we closely monitor the investment decisions and performance outcomes of all subgroups in the treatment group, as well as the control group. The key outcome variables include changes in portfolio composition, trading activity, investment performance, risk management, financial literacy improvement, behavioral changes, and international diversification.

This experimental design allows us to rigorously evaluate the causal impact of financial literacy on investor behavior and performance, providing valuable insights into how educational interventions can enhance investment outcomes. The results of this trial will contribute to the broader understanding of financial literacy's role in investment decision-making and its potential benefits for investors.

Not available

Randomization Method Randomization for our study was conducted using a computer algorithm to ensure transparency and reproducibility. We utilized Stata code to automatically randomize and re-randomize participants as follows:

Initial Allocation: We first allocated the 28,819 investors to the treatment and control groups based on a randomly generated number. This initial random allocation was performed to ensure each investor had an equal probability of being assigned to either group.

Balance Check: After the initial allocation, we calculated the maximum and average t-statistics for the differences in key baseline characteristics between the treatment and control groups. The variables used for this balance check included:

Investors' age Gender Years of schooling Total asset value at the brokerage firm Profit/Loss (P/L) in terms of percentage in the past year

Re-randomization Process: If the maximum t-statistic for these variables exceeded 1.25 or the average t-statistic was higher than 0.35, we drew a new random number and reallocated investors to the treatment and control groups based on this new number. This re-randomization process was repeated until the maximum t-statistic was 1.25 or lower and the average t-statistic was 0.35 or lower. This ensured that the treatment and control groups were well-balanced in terms of these key characteristics.

Final Allocation: Once the balance criteria were met, the final allocation was used for the study. This method ensured that any differences in outcomes between the treatment and control groups could be attributed to the financial literacy intervention rather than pre-existing differences between the groups.

Investors

No

30,000 investors

30,000 investors

30,000 investors