Исследование показало, что готовые языковые модели не могут точно имитировать человеческие риски

Large language models (LLMs) are increasingly used as agents to simulate human behaviour, yet their fidelity in complex decision-making under uncertainty remains insufficiently understood. To address this gap, researchers in China have developed a comparative framework that benchmarks LLM-simulated risk preferences against empirical human behavior. Using demographic profiles from surveys conducted in Sydney, Hong Kong, and Nanjing, the team constructed role-playing prompts and evaluated three LLMs on abstract lottery-choice tasks. They adopted the classical Constant Relative Risk Aversion (CRRA) framework as a domain-neutral “standard ruler” to compare risk attitudes. They found that off-the-shelf LLMs do not exhibit a universal risk profile: the two GPT models are more risk-averse than human benchmarks, whereas Gemini is more risk-seeking. They said prompt language systematically affects simulated risk attitudes, with English-to-Chinese switching inducing a more conservative shift in most cases. Also, LLMs do not reliably reproduce the empirical heterogeneity of human risk preferences, tending either to generate overly concentrated distributions or unrealistically large dispersion. These findings, published in the journal Communications In Transportation Research, show that off-the-shelf LLMs remain vulnerable to model-family-specific miscalibration, language-sensitive distortions, and failures in distributional fidelity. Rigorous empirical calibration is therefore necessary before off-the-shelf LLMs can be reliably deployed in computational social science and choice modelling. The findings highlight an important limitation of using off-the-shelf LLMs as tools for behavioral prediction. Since different model families exhibit different baseline calibration biases, the choice of model can materially affect the inferred pattern of public risk preferences. In practice, one model family may overstate conservatism, whereas another may overstate willingness to accept risk. Without empirical calibration, such biases can distort inference and lead researchers to draw policy conclusions that do not accurately reflect observed human behaviour. This limitation is particularly consequential in transportation research, where risk perception is central to decision-making under uncertainty. Travel behaviour frequently involves probabilistic trade-offs, including route choice under unreliable travel times, mode switching during service disruptions, and the adoption of emerging mobility technologies under safety and performance uncertainty. If the underlying risk preference parameters are systematically miscalibrated, demand forecasts, welfare evaluation, and policy design may all be biased. For example, using uncalibrated LLM-generated data to infer willingness to adopt safety-critical systems such as autonomous vehicles or low-altitude mobility services could yield either overly conservative or overly optimistic projections, depending on the model family used. The multilingual results add a further layer of caution. In linguistically diverse settings, prompt language is not a neutral implementation choice: it can systematically perturb the behavioral calibration of the model. This is especially relevant for transportation systems serving multilingual populations, where researchers may be tempted to use native-language prompting as a straightforward way to improve realism. ”Our results suggest that such an assumption is unwarranted unless the model has first been validated against human benchmarks in the relevant linguistic context,” the researchers said. “Overall, the implications of this study are methodological as much as substantive. Off-the-shelf LLMs should not be treated as direct substitutes for human respondents in risk-sensitive behavioral applications,” they said. “Instead, they should be regarded as tools whose outputs require domain-specific and language-sensitive calibration. Rigorous validation against human ground truth remains a necessary prerequisite for deploying these models in transportation and other cross-cultural social science settings, according to the researchers.