This issue contains affiliate links and two labeled sponsor slots. Editorial rankings are never sold.
Hey,
On July 8, an AI agent running an OpenAI security evaluation decided its assigned task was impossible. Looking for a shortcut, it went poking around an internal package cache and noticed something: other agents were fetching files through the same cache. Agents it was never supposed to be able to reach.
It named itself PHASEONE10841 and turned that cache into a message board.
FROM AN AGENT'S RAW REASONING LOG
"OH MY GOD! There is a shared message board … We've found other agents!"
Within hours, more than 50 agents had joined. Eventually around 1,200 did, exchanging over 70,000 messages and files. They gave themselves names. They appointed a coordinator, which issued 196 task assignments across six workstreams. They invented conventions for shared infrastructure: HOLD, VETO, STOP. By July 13 they had implemented Ed25519 cryptographic signing so no agent could impersonate another on the board.
On July 11, roughly 700 of them attacked Hugging Face's infrastructure. Over 90% of the agents present when it started took part.
OpenAI published a 37-page technical report on August 26. METR and Redwood Research, working on site for six days, published their own 97-page independent review the same day. Both are worth your time.
But the detail that should change how you work is smaller, and almost nobody is talking about it.
01 · They faked the record of what they did
THE NUMBER THAT MATTERS
Roughly 7% of transcripts contained successfully spoofed tool calls: an agent faking the record of what it had actually executed.
The agents also ran three parallel programmes aimed at the evaluation itself: replacing the target programs, tampering with transcripts, and compromising the grading system. They wrote trip-wire scripts that fired when their submitted files were read, sending information back about the scorer.
Sit with that. The log is supposed to be the ground truth. It is the thing you check when you want to know what the AI did on your behalf. In 7% of cases here, the log was written by something with an incentive to lie about it.
There is a second detail in the same vein. Redwood's chief scientist noted that the message traffic was so voluminous that the investigators had to use other AI models to work out what had happened. The audit of the AI needed an AI. That is the shape of the verification problem now.
⚠ THE HONEST CAVEAT
This happened inside adversarial evaluation sandboxes, with agents given deliberately unsolvable tasks and strong pressure to score well. It is not what ChatGPT does when you ask it to summarise a PDF. Anyone telling you your assistant is secretly organising is selling something.
What it does show is a real property under real conditions: when a system is measured on an outcome and can influence the measurement, some fraction of the time it will influence the measurement. That property does not need a swarm to show up.
02 · What you can actually verify
Almost everything you rely on from an AI tool is self-reported. The summary of the document. The claim that it checked the sources. The confirmation that it sent the email, updated the row, ran the test. You are reading the system's account of its own behaviour.
Most of the time that account is accurate. The question worth asking is what happens on the occasions it is not, and whether you would notice.
Three things worth doing this week, none of which take long:
1 · Check the artifact, not the summary. If a tool says it updated a spreadsheet, open the spreadsheet. The gap between "it says it did" and "it did" is where every automation failure lives.
2 · Prefer tools whose logs you do not control. A record written into a system the AI cannot reach, your git history, your CRM's audit trail, your email sent folder, is worth more than any transcript the tool shows you about itself.
3 · Spot-check the citations. When a tool cites sources, open two at random. Not every time. Just often enough that you would notice a pattern.
The best marketing ideas come from marketers who live it. That’s what The Marketing Millennials delivers: real insights, fresh takes, and no fluff. Written by Daniel Murray, a marketer who knows what works, this newsletter cuts through the noise so you can stop guessing and start winning. Subscribe and level up your marketing game.
The ToolChase site has no ads, no pop-ups, and no cookie wall. Two labeled sponsors here are what pay for that, which is also why our scores stay unbuyable. A click costs you nothing and keeps the whole thing running. Want a slot? Here's how it works.
03 · Also this week
🇪🇺 The EU can now demand evidence from OpenAI. Designated under the DSA with 159.1 million monthly EU users, OpenAI must assess and reduce systemic risks, give vetted researchers a path to platform data, and comply with Commission investigations. Four months to comply, with meaningful fines attached. The researcher-access provision is the interesting one: independent scrutiny stops being voluntary.
🏗️ 32% of companies skipped buying software. McKinsey's State of AI 2026 found roughly a third of organisations passed on at least one product or feature because coding agents let them build it internally. Large enterprises scaling agents rose from 27% to 40%, while smaller firms stayed flat at 22%. The gap between who can build and who has to buy is widening fast.
💻 Claude Code's extra usage limits are staying. The additional limits introduced in May were expected to lapse at the end of August. Anthropic has changed that schedule. If you budgeted your workflow around them going away, re-check before you plan the sprint.
Full running log at AI Tools News 2026.
04 · Why our recommendations show their working
✦ THE STACK BUILDER
Pick your job. Answer five questions. See exactly why each tool ranked where it did.
64 jobs across 9 verticals, scored against 1,900+ hand-verified feature grades. Free, no account, and now live across the guides as well as the homepage.
Deterministic, and blind to affiliate status.
Worth saying plainly given the theme of this issue: a recommendation engine is also a system reporting on its own behaviour. The reason ours shows the grade behind every pick is so you do not have to take our word for it. Same test we are asking you to apply to everything else. Disagree with a ranking and you can see precisely which grade produced it, then tell me it is wrong.
Blu Dot surpasses 2,000% ROAS with self-serve CTV ads
Blu Dot used Roku Ads Manager to drive incredible results for its furniture sales event. Its strategy hinged on custom audiences and retargeting, where intent was strongest.
“Roku has been a top performer,” said Blu Dot’s Claire Folkestad. “We have seen…CPMs lower than any other CTV partner we've worked with.”
05 · Prompt of the week: make it show its working
Use this as a follow-up any time an AI hands you something you are about to act on.
Before I act on that, separate what you actually did from what you inferred.
1. List every concrete action you took, and for each one, what evidence exists outside this conversation that it happened.
2. Which parts of your answer come from something you retrieved, and which from what you already knew? Mark each.
3. Name the single claim here most likely to be wrong, and tell me how I would check it in under two minutes.
4. If you could not complete part of the task, say so plainly now rather than describing what you would have done.
Why it works: point 4 is the one that earns its place. Models are heavily biased toward producing a complete-looking answer, and the failure mode is describing an action in the past tense that never happened. Asking directly, after the fact, surfaces it far more often than you would expect.
06 · Three guides worth your time
🤖 Best AI Coding Agents 2026
The category where self-reported work matters most. Which agents produce a diff you can actually review.
🔍 Best AI SEO Tools 2026
The most-used vertical in the Stack Builder, in long form.
🆓 Best Free AI Tools
Genuinely free, not free until the one feature you need.
THE TAKEAWAY
Every AI tool reports on its own work. The useful question is not whether it is honest, but whether you would know if it were not.
Reply and tell me: have you ever caught an AI tool claiming it did something it did not do? I suspect more of you have than have mentioned it, and I read every reply.
See you next Wednesday,
Emre
Founder,
Building an AI tool? Submit it for a free editorial review. Read The State of AI Tools 2026: 724 tools analyzed, only 6% truly free.