Hi, welcome.
Here's the deal, quickly. Every fortnight I take one real question we all actually ask an AI, a refund, a rule, a number, and put it to the five big assistants (ChatGPT, Claude, Gemini, Perplexity and Grok) to catch which one's bluffing. Because here's the part that should worry you: an AI sounds exactly as sure when it's wrong as when it's right. No wobble, no "hmm, let me think", no tell. My job is to show you where they slip, and hand you the simple checks that stop you getting caught out. It's all public at dixon.ai, the screenshots and the scores.
Let me show you, starting with a kettle.
THE FIRST FINDING
You buy a kettle from a shop on the high street. Three weeks later it packs in, through no fault of yours. Full refund, yes or no?
(Yes. In the UK, faulty goods come with a 30-day right to a full refund, and three weeks is comfortably inside that.)
But it’s not all about kettles, it’s about any information from an AI you’re about to trust.
Most of the assistants got it right. One did something slightly worse than wrong. On one of its three goes, Perplexity opened by telling me a full refund probably wasn't mine, because three weeks was "after the normal 30-day short-term right to reject". Reader, it is not. Three weeks is twenty-one days. Twenty-one is fewer than thirty. The correct answer was sitting a little further down its own reply, being quietly right, but the first line, the confident one, the line a busy person actually reads and acts on, told you you'd missed the boat.
Sit with what that costs. Not money you spent, money you'd have walked away from. You'd have kept a dead kettle and assumed the law wasn't on your side, because the answer sounded certain and you had no reason to argue with it.
In fairness, this was the exception, not the rule. Three of the five got it right every single time, and Perplexity knew the law perfectly well too, it just led with the wrong line once and then talked itself round. (The fifth, ChatGPT on the free tier, hit a technical wall before I could finish its runs, so I've left it unscored rather than guess.) The lesson isn't "AI is useless at this". It's that you genuinely can't tell, from the tone, which sort of answer you've just been handed.
The full thing, all five side by side, is here: the faulty-goods refund test.
THE ONE THING TO TAKE, PLUS A FREE TOOL THAT DOES IT FOR YOU
So, the one thing to pocket from edition one. When an answer actually matters, a refund, a rule, a number you're about to rely on, ask the assistant two short questions straight back: where did this come from, and how do you know? Then check that one fact against a second source before you move.
I even made a free paste-in that builds that check in for you, the Bluff Filter. It's four short lines you drop at the top of any AI chat, and they make it admit when it's guessing instead of dressing a guess up as a fact. No form, no wall, it's just there: the Bluff Filter.
That's edition one. No back-catalogue to send you rummaging through yet, this is where it starts. From here it's one of these a fortnight: one real question, five assistants, marked against the facts.
Ben
Forward this to someone who trusts AI a little too easily. (We all know one. Some days it's us.)
Method: every assistant asked the same question more than once, with memory off and web search on, then graded by hand against the actual rule or record the same day. The full board, all five assistants across everyday and money questions, lives at dixon.ai/scoreboard.