A cross-cultural comparison finds popular language models misjudge human risk attitudes in opposite directions, and shift depending on the language of the prompt.

Surveys are expensive, slow and increasingly hard to fill. So it is perhaps no surprise that researchers in fields from marketing to transportation planning have begun asking whether large language models could stand in for human respondents. Give an AI a demographic profile, ask it to role-play that person, and record its answers. If the results matched real people, the savings would be enormous.

A new study in Communications in Transportation Research tests that idea on one of the most consequential aspects of human decision-making: how people handle risk. Its answer is a firm caution. Off-the-shelf language models, the authors conclude, should not be treated as direct substitutes for human respondents in risk-sensitive applications.

Measuring risk with a standard ruler

Economists have long measured attitudes toward risk using lotteries. Would you rather have a guaranteed sum, or a coin flip for a larger one? By presenting people with a series of such choices, researchers can estimate how risk-averse or risk-seeking they are. A common way to summarize the result is the constant relative risk aversion, or CRRA, framework, which boils a person’s pattern of choices down to a single parameter.

The researchers used CRRA as what the release calls a standard ruler. They took demographic profiles from surveys of real people in three cities, Sydney, Hong Kong and Nanjing, and built role-playing prompts asking language models to answer as those individuals would. Three models, two from OpenAI’s GPT family and one from Google’s Gemini, then completed the same abstract lottery-choice tasks, and their estimated risk attitudes were compared with those of the real human respondents.

To see what the lottery tasks capture, consider a simple choice: a guaranteed $50, or a 50-50 chance at $100 or nothing. The two options have the same average payoff, so a person who takes the sure thing is showing risk aversion, while one who flips the coin is showing a taste for risk. By varying the amounts and odds across many such questions, researchers can locate each respondent on a scale. In this study, the question was whether a model told to imagine itself as, say, a middle-aged commuter in Nanjing would land in the same place on that scale as real middle-aged commuters in Nanjing.

Three problems

The comparison revealed three distinct problems. First, the models lacked any universal risk profile. The two GPT models came out more risk-averse than their human benchmarks, while Gemini was more risk-seeking. A researcher who relied on one model family might conclude that the public is cautious; one who relied on another might conclude the opposite.

Second, the language of the prompt mattered. Switching from English to Chinese prompts systematically shifted the models’ simulated risk attitudes, usually toward greater caution. That is a significant finding for multilingual settings like Hong Kong, because it means the choice of prompt language is not a neutral technical detail.

Third, the models struggled to reproduce the variety of real people. Human populations contain a wide spread of risk attitudes, from the very cautious to the adventurous. The simulated respondents produced either distributions that were too tightly bunched, as though everyone thought alike, or a spread that was unrealistically wide.

Why transportation cares

Risk perception sits at the heart of how people travel. Commuters choose routes knowing travel times are unreliable, decide whether to switch to another mode when a service is disrupted and weigh whether to try new technologies. Planners build those choices into demand forecasts, welfare evaluations and policy decisions.

If simulated respondents systematically misjudge risk, those downstream calculations inherit the error. The study points to emerging technologies such as autonomous vehicles and low-altitude mobility services, including drone delivery and air taxis, as cases where the stakes are high. Depending on which model family generated the simulated data, projections of public willingness to adopt such services could come out overly conservative or overly optimistic, potentially misdirecting investment and regulation.

The authors also warn against a tempting assumption: that prompting a model in a population’s native language will automatically make its answers more realistic. Unless the model has been validated against human benchmarks in that language, they argue, there is no guarantee of improved fidelity.

Tools, not stand-ins

The study does not argue that language models are useless for behavioral research. Instead, it reframes them as tools that require domain-specific and language-sensitive calibration, checked against real human data, before their outputs can be trusted. Rigorous validation against human ground truth, the authors conclude, remains a necessary prerequisite.

That conclusion fits a broader pattern emerging from research on so-called silicon samples, in which language models are used to simulate survey respondents. Some studies have found that models can reproduce average opinions reasonably well, but they tend to flatten the diversity of real populations and can carry systematic biases rooted in their training data. Risk attitudes, which depend on personal experience, culture and circumstance, may be especially hard to simulate.

The release leaves several questions open. It does not report how many human respondents or simulated runs were included, give the specific model versions, or report numerical estimates of how far the models diverged from humans. Model behavior also changes between versions, so results for one generation of systems may not hold for the next.

The expert contact for the study is Chenyang Wu of Northwestern Polytechnical University in China. For now, the takeaway for anyone tempted to replace a survey with a chatbot is straightforward: check the machine’s answers against people first.

---

Notes

1. “Can large language models capture human risk preferences? A cross-cultural study.” Communications in Transportation Research, 30 June 2026. DOI: 10.26599/COMMTR.2026.9640025.
2. Tsinghua University Press. “Can large language models capture human risk preferences?” EurekAlert!, 23 July 2026. https://www.eurekalert.org/news-releases/1137238.