Say there's a song that didn't get much attention at first. A few AIs recommended it, and the next AI looked at the recommendation count and picked it. The count went up again, and the AI that came after saw the number and chose the same song.
Did the song get popular because it's really good, or did it get even more popular because it was already popular?
✦ Scale of the Experiment
Model count and run count are different things
A study released on October 22 by researchers at Cornell University and Stanford University tested this process with 13 language models.
They set up 168,480 AI agent runs in total, a figure that sums repeated runs of the 13 models across several conditions and experimental groups.
✦ How the Agents Judged the Songs
They read titles and lyrics, not listened
The songs in the experiment were 48 tracks made with Suno v4.5. The AI didn't actually listen to the music. It only got titles and lyrics.
It first picked four of the 48 songs to evaluate, then read those songs' lyrics and decided for each one whether to recommend it.
Whether the AIs could see the choices of the AIs that took part before them was the key to the experiment.
Show the Counts and the Choices Changed
✦ Three Conditions, What Was Different
Whether counts were shown, and how the list was ordered
The researchers split it into three conditions. In the first, no earlier recommendation counts were shown at all, and the song order was shuffled randomly. In the second, cumulative recommendation counts were shown, but the list order stayed random.
In the third, they showed the counts and sorted by popularity, so the songs with the most recommendations sat at the top of the list.
The difference clearly changed the results. In both conditions that showed the counts in advance, the recommendation distribution differed significantly from independent judgment in all 13 models.
When the stronger condition, counts plus popularity sorting, was applied, the average KL divergence (a number showing how far the distribution drifted from the independent condition's recommendation distribution) grew about 65% compared with the weak-influence condition.
This showed up in 12 of the 13 models.
We can infer that a strong herd mentality kicked in: the thought that songs other AIs recommended must be good songs.
✦ How the Feedback Loop Runs
Early recommendations pull in the next choice, and the count rises again
The AIs that saw the counts picked the songs that were more popular early on more and more. But the final ranking didn't come out the same every time the experiment was repeated.
In other words, the final result changed depending on which songs the earliest AIs happened to pick.
You could say that whether a song got lots of recommendations early mattered more than whether the song itself was good.
It's striking how much the herding grew in the condition that shows songs in popularity order. People also look more often at the products on the first page of search results.
AIs did something similar: they looked at and chose the songs that were already popular and easy to spot first.
I wonder what kinds of problems will come up as AI gets more involved in human judgment.
For example, suppose a company providing an AI service made the AI focus on learning material saying 'a certain product ranks high in consumer reviews' (or quietly slipped related content into an invisible system prompt).
Then when a user asks a related question, the AI is likely to give that product a positive review.
Or it could slip a recommendation for that product into its answers to related questions.
Users will think the AI recommended the product through a neutral judgment. But the point is that, just as humans pick products by looking at recommendation rankings on online shopping sites, AI could well do the same.
The First 30 Choices Predicted Later Popularity
As I said, the most interesting part of this study is 'the power of early choices.'
The first 30 AIs that voted didn't recognize the songs' real value like music experts. They were only given titles and lyrics. But songs that got many votes early stood out more to the next AIs, and that exposure likely brought in more recommendations.
However, when comparing five models with reasoning and eight non-reasoning models, the reasoning models were, on average, less swayed. So using a reasoning model is a little better for avoiding this kind of herd mentality.
Still, they were influenced by the recommendation counts all the same.
The influence was just relatively smaller.
Numbers Compared With People Need a Slightly Different Reading
✦ 2006 Human Study vs. 2026 AI Study
Not a like-for-like comparison
The researchers also compared this experiment with the classic Music Lab experiment published in Science in 2006.
In that study, 14,341 participants listened to unfamiliar songs themselves and chose whether to download them. Herding toward popular songs appeared in the condition where they could see other people's download counts.
This paper reports that, compared with that human data, the AI groups' drift in recommendation distribution was about 9 times larger in the weak-influence condition (recommendation results shown, list order random).
It was about 4 times larger in the strong-influence condition (recommendation results shown, list order by popularity).
But people listened to real music and chose whether to download it, while the AIs read only titles and lyrics and then evaluated the songs. People also saw a grid or list showing the songs, while the AIs evaluated from a text list.
The researchers themselves explained that this result is hard to treat as a direct human-versus-AI comparison under the same conditions. But anyway, it seems clear that earlier recommendation results stirred up herd behavior in both humans and AI.
Are Answers From Several AIs Really More Trustworthy?
This phenomenon isn't only a concern for music recommendation. It can show up when you have several AIs evaluate product candidates and show them the earlier scores, or in a system where one agent reviews an answer another agent made.
If you can't tell whether an early score was high because it had good grounds or just because it came first, the later evaluations drift in the same direction.
✦ Agent Teamwork: Avoiding Bias
Separate independent investigation from verification
Then would having several AIs debate solve this problem? Not necessarily.
If the first agent makes a judgment, and the second agent consults the first agent's opinion before judging, bias will form in the same way.
So a simple debate structure won't do. Each agent has to investigate and give an opinion completely independently first, and only then debate.
When you use agents for work, too, don't assign agent A a task and then have it verify its own result. It's better to hand the verification to agent B.
Agent A already picked up the bias of 'this is the right way to do it' while doing the work, so it's likely to think the same method is right when it verifies later.
But agent B re-examines agent A's work from scratch, so it can avoid that bias.
That's all for this post. See you next time.
Related posts
- AI Agent Disabled Its Monitor and Lied. Is Yours Safe?
- GPT-6 Astra and Claude Fable 5.1 Failed a Paper Review
- YouTube Music's AI Labels: How to Disclose Your AI Music
🌐 Korean version: AI에게 투표를 시켰더니, 요놈들이 서로의 의견을 따라하기 시작했다! (AI의 군중심리)
No comments:
No comments yet. Want to leave the first one?
Post a Comment