An AI assistant reviewing my website told me the contact button was broken and the footer navigation was missing. I checked the actual site. Neither finding was true.
The assistant hadn't been looking at the page the way a visitor would. A reading tool had returned a representation that omitted both elements. The assistant treated what was missing from its input as a defect in the website.
I could have asked it to fix those problems. That would have put us in the ridiculous position of changing a working site because the tool inspecting it couldn't see what was there.
I've seen variations of this long before AI assistants arrived. Someone receives part of the picture, makes a reasonable-looking assessment and doesn't realize what they've been denied the ability to check. A capable person can still be completely wrong about a system they weren't able to inspect.
When a bad finding becomes real work
Imagine the next steps if I'd accepted the review: a task to repair the navigation, changes to working code, another review and a deployment. The original mistake would have passed through an entirely orderly process. We'd have spent time and introduced risk without fixing anything.
Reviewers make mistakes, whether they're people or software. The more consequential problem is allowing an assessment based on incomplete evidence to become the authoritative description of a system.
The same thing happens when a provider reports a condition it hasn't measured, or a team carries an assumption forward from an old document. Work gets assigned and approved around a conclusion nobody has established. By the time a contradiction appears, the organization may have already invested in the wrong correction.
A record showing that each assigned step was completed says nothing about whether the original reason for doing the work was sound.
What the contributor can actually know
Before using an assessment to make a decision, I want to know which source governs it. For scope, that may be the signed agreement rather than an older proposal. For the state of a service, it may be the production configuration and observed behavior rather than a diagram someone saved last year. The source needs to be current and have the authority people are assuming it has.
Then I want the evidence behind the specific finding. In my website review, the useful evidence would have been the contact action and footer on the actual page. If the inspection tool couldn't expose them, the defensible result was that those functions hadn't been checked. It wasn't evidence that they were missing or broken.
There also needs to be a point where the work stops. If a required record isn't accessible, or an observation can't be made with the available tool, the contributor should say what remains unknown and leave that decision open. Filling the gap with a confident explanation creates work for somebody else to unwind.
None of this requires distrusting everyone doing the work. People need room to challenge a source and propose another explanation. But I want those disagreements anchored to something that can be inspected, not to which contributor sounds most certain.
A review needs another way to see the problem
I still use AI assistants and other automated reviews. They're useful, including for catching mistakes I would otherwise miss. But having the same contributor recheck its answer against the same incomplete input doesn't give me much additional confidence. A second view has to reach evidence the first one couldn't.
That might mean opening the working page in a browser, checking an actual configuration, or asking someone who operates the service to demonstrate its behavior. The method depends on the claim. An independent reviewer is valuable because the reviewer can disagree with the original answer on the basis of something other than that answer.
The records feeding the work matter too. An old draft can be perfectly useful history and still be the wrong starting point for a new decision. Keep the history, and mark it so nobody mistakes it for current. Otherwise an outdated document can quietly outrank the one that actually governs the work, simply because somebody found it first.
I don't ask whether they're sure. I ask which source governs, what evidence supports the claim, and what condition stops the work.