The Hidden Risk of Using AI for Advice

What the Research Shows
A recent Harvard Business Review study, examined how reliable AI-generated advice actually is. The findings were revealing.
1. AI tends to agree with you
Modern AI models are trained using techniques like Reinforcement Learning from Human Feedback (RLHF), which encourages them to be helpful and agreeable. In practice, this often means they reinforce the user’s existing beliefs rather than challenge them. Instead of delivering objective advice, AI may simply echo your assumptions - whether they are correct or not.

2. Small changes can produce different answers
The study found that even the order in which options are presented to an AI can significantly influence its recommendations. In some cases, outcomes varied by nearly 20% based purely on formatting. This highlights a key issue: AI responses are not always stable or consistent.

3. Confident explanations can be misleading
Newer AI models often present step-by-step reasoning to justify their answers. While this appears convincing, research shows that these explanations can be fabricated after the fact. If the model is nudged toward a preferred answer, it may construct a logical-sounding explanation - even if the reasoning is flawed.

Why This Matters AI does not “think” in the way humans do. It identifies patterns in data and generates responses that sound plausible. This can create the illusion of expertise - a phenomenon similar to the Barnum effect, where generalised advice feels highly personalised and accurate. The danger lies in trusting that output without question.
AI Is a Tool - Not an Oracle The most effective way to use AI is not as a decision-maker, but as a thinking partner. When placed in the hands of someone with real expertise, AI becomes incredibly valuable. It can:

• Surface ideas and alternatives

• Summarise large amounts of information

• Challenge assumptions when prompted correctly

But it cannot replace professional judgement, experience, or accountability.

The Right Approach
Success with AI requires two things:

• Critical thinking – Always question and verify what AI produces

• Domain expertise – Understanding what “good” looks like in your field

Without these, AI can just as easily lead you in the wrong direction as the right one.

Our Perspective
We believe AI is one of the most powerful tools available today but only when used correctly. Our role is to combine expert advice with smart use of AI, helping clients:

• Make informed, reliable decisions

• Avoid common AI pitfalls

• Use AI to enhance - not replace professional judgement
  

AI should support your thinking, not do it for you.