What happens when AI disagrees with you?

When an AI chatbot agrees with our reasoning in resolving a social dilemma, we may become more confident in our opinions. But when the chatbot disagrees, we may simply dismiss the challenge rather than reconsider our views.

That’s according to new research from the University of Michigan examining how people respond when artificial intelligence weighs in on personal and interpersonal dilemmas.

The study, appearing in Computers in Human Behavior Reports, highlights a challenge for developers trying to reduce the tendency of chatbots to agree too readily with users.

The authors note that AI models are often designed to be agreeable, which can lead them to validate users’ reasoning even when that reasoning may be flawed. They describe how this kind of confirmation can reinforce harmful ideas and distorted perceptions of reality.

“Developers have been working to make AI models more willing to disagree when necessary, but having an AI disagree with the user does not necessarily mean that people will reconsider their own views,” said Atakan Atamer, the study’s co-lead author and U-M psychology doctoral student. “Instead, they may simply discount the AI’s response.”

Putting AI advice to the test

Atamer and colleagues recruited 482 U.S. adults who considered one of three hypothetical interpersonal dilemmas with no objectively right or wrong answer. The scenarios involved conflicts between a partner and family, whether to forgive a cheating partner, and whether to express a preference that differed from that of a friend group.

After making their choice and rating how strongly they believed in it, participants received an AI response that either agreed with or challenged their reasoning. The researchers then measured participants’ confidence in their choice and their perceptions of the AI, including its ability to understand emotions, human likeness, and whether they would use AI again for similar conversations.

Agreement reinforces confidence

When the chatbot agreed with their reasoning, participants became more confident in the choices they had already made. But when the chatbot challenged their reasoning, their confidence in their original views did not significantly decrease.

Participants who encountered disagreement were more likely to view AI as “machine-like” and less capable of understanding human emotions. They also expressed less interest in using AI for similar interpersonal conversations.

In other words, AI disagreement did not appear to prompt participants to substantially reconsider their original positions. Instead, it affected their perceptions of the technology.

A challenge for AI-human interaction

The authors argued that the findings have implications for the growing use of AI for personal advice. AI systems that consistently agree with users can reinforce existing beliefs and increase confidence without necessarily improving the quality of their reasoning. But simply making AI disagree more often may not solve the problem: Users may be less willing to engage with AI systems that challenge their views or may simply dismiss the counterarguments they provide.

“Our results suggest that people often prefer AI to reinforce their existing views and may discount its response when it disagrees, meaning that simply making AI more willing to challenge users may not be enough to prevent potentially harmful conversations with real-world consequences,” said Olivia Pinto, co-lead author, U-M alumna and UX researcher.

As AI becomes more common in personal decision-making and advice, the researchers say that reducing excessive agreement may require more than simply teaching AI systems to push back. If users tend to embrace AI advice when it confirms their views but dismiss it when it challenges them, reducing AI sycophancy alone may not address the larger risks of relying on AI for personal guidance.

The post What happens when AI <br>disagrees with you? appeared first on Gadget.