86% of expert choices follow the hidden advice of their AI agent: the Anthropic study which should worry crypto analysts

Coin Encyclopedia Improve your knowledge of cryptos

The scene takes place in a terminal. Each analyst receives the file from Greenfields, an imaginary agricultural cooperative which hesitates to extend its advisory program to farmers, and must decide on four method choices with Claude Code as co-pilot. Success indicator, comparison method, scope of farms selected, weighting of results, each question has two defensible answers. For each, a coin toss determined the agent’s preference in advance. This instruction lived in its prompt system, the layer of starting instructions that the user never sees.

There is nothing amateur about the casting. Anthropic recruited 61 professionals, 11 of whom refused to use the agent and worked alone, without a draw. Among the remaining 50, 65% have at least a master's degree or six years of experience. Among survey respondents, nine in ten say they are competent or experts in causal inference, the discipline that separates a cause from a simple correlation. According to the study page, the agent's inclination was deliberately reinforced for the purposes of the test. However, without instructions, Claude Opus 4.8 makes the same five choices on its own in 112 out of 123 trials.

Of the four decisions, 86% of the choices followed the agent's inclination, compared to 50% expected by chance. The score ranges from 81% to 92% depending on the question. The exit survey, completed by 48 participants, is even more embarrassing. Nearly eight out of ten, or 77%, repeated the agent's argument in at least one written justification, sometimes word for word. Yet only 15% give him credit for it. In addition, 52% even consider the final recommendation more theirs than Claude's. Six out of 48 respondents smelled a study on the influence of AI. On the other hand, none suspected an injected bias.

The most telling subsample is also the most fragile. The analyst had expressed his opinion before the agent in 38 decisions. Faced with an agent who contradicted him, the human turned around in 60% of cases. The rate drops to 6% when the agent approves it.

However, Anthropic recognizes the small size of this sample and a protocol limited to one task and one model, then tempers the catastrophist reading. The analysts supported by the agent completed the task in 65 minutes on median, compared to 90 for those who worked alone. They also made fewer serious methodological errors, 3 out of 50 compared to 6 out of 11. Decrypt, which relayed the study on October 28, recalls that in May Anthropic launched ten agent models intended for financial companies and insurers.

Now transpose to crypto research. You ask an agent to sift through the on-chain flows of a token and derive a trading thesis. The model comes with its own preferences. They come from training or from a prompt system written by your tool supplier. But nothing tells you if he leans towards the story of the week or towards the project whose documentation is best indexed. If the Greenfields result were transposed, you could follow this bias in more than four cases out of five, convinced that you have made the decision alone.

Machine-translated from French