When the Agent Is Wrong and Sounds Right

Most people who work with AI agents will tell you they "always double-check the answer." Most people are not actually doing it. The disclaimer at the bottom of every chat interface is read once and forgotten. The answer comes back fast, fluent, and confident. It sounds correct. You move on.

This is a post about a specific incident where I almost did the same thing, and what stopped me. The lesson is not new. Anyone who has been working with these tools for more than six months has run into a version of it. But the cost of not internalising the lesson is high enough that I think it is worth telling the story in detail.

The setup

I have been building a new platform integration that I am not ready to talk about yet. It is in active development and the surface area is changing daily. What matters for this post is that the work required setting up a Meta developer account and wiring up webhooks against the Meta platform. The product is originally targeting a market with significant payment and platform restrictions, and the current plan is to launch first to a broader audience and bring it back to the original market when the conditions allow.

For context, Meta runs one of the most comprehensive developer platforms in the industry. Their dashboard gives you fine-grained access to test almost every feature you might build against. Tokens, permissions, app review states, sandbox environments, simulators for events you cannot easily generate yourself. They have done a lot of thoughtful work to let developers verify their integrations before submitting anything for production. That is going to matter later.

The project itself is simple at this stage. Two test cases. The first one: when a user sends a direct message to the test account, the system replies with an automated response. The second one: when a user comments on a post, the system fires a webhook that I can respond to. This is a standard pattern. Anyone who has scripted Instagram or Facebook engagement has built something close to it.

The first test worked the first time. I sent "this is a test" from a sandbox account to the main account. The response came back immediately. The integration was real. Everything looked right.

Then I tried the second test. Nothing.

What the agent told me

I went back to the AI agent I was working with and asked what was wrong. The webhook was not firing. The comment had been posted. The dashboard said the webhook subscription was active. ngrok was watching the endpoint and seeing no traffic.

The agent had an answer ready. "You need to submit the product for review before this kind of webhook will fire."

This is the kind of answer that sounds right. Submission for review is a real thing in the Meta platform. Some features do require it. The phrasing was confident and specific. If I had been a year less experienced, I would have nodded, started writing a submission, and given up trying to test locally.

Instead I pushed back. The thing the agent was telling me did not match what I could see. I had something that was, generously, fifty lines of code. There was nothing to submit. Submitting fifty lines of code with one rule on it as a "product" to Meta would have been a comedy. They would not have read the first line. And more practically, I could see in the dashboard that there was a section explicitly built for testing webhooks before submission. I did not know how to use it yet, but it was sitting there in the UI.

I went back to the agent. I said this does not match what I can see. The dashboard has a webhook test panel. There is no way the platform would build a comprehensive test harness and then require me to submit the product before testing the webhook part of it. That defeats the whole point.

Three rounds of pushback

The agent's first response to my pushback was not great. It restated the previous answer with slightly different wording. Submission required, this is just how it works in test mode, the dashboard panel is for something else.

I pushed again. I told it explicitly: do not rely on your training data. The platform has changed. Go and research. Look for documentation that is current. Look for forum threads about the webhook test panel.

The second round came back with more nuance but still did not solve the problem. Some features can be tested without submission, the agent said, but you need a token with the right scopes, and getting the right scopes for the personal account I was testing from would require submission.

I pushed a third time. I said the platform is not designed to require submission for a test. There is something I am missing about the test flow. Find it.

The third round came back with the answer. Meta has a webhook test panel inside the developer dashboard that lets you fire a synthetic event from the platform itself, against your webhook URL, with a token you generate from the dashboard. No submission required. You configure the subscription, you click test, the platform fires the event at your endpoint, you see whether it lands. The whole point of the panel is to verify the integration before you ever ship.

I configured it. The synthetic event fired. ngrok caught it. The webhook handler responded. The integration was working all along. The thing the agent had been telling me, three times, was simply wrong.

Why this matters

It would be easy to read this story as "the agent is unreliable." That is not the lesson I want to land on, because it makes it sound like the answer is to use AI agents less. The answer is the opposite. The answer is to use them well, and using them well includes knowing the shape of their failure modes.

The specific failure mode here is that the agent reached for its training data first. Training data has a cutoff. Platforms change. Documentation moves. Features that did not exist when the model was trained, or were undocumented at the time, will be invisible to the agent until you force it to look at the current state of the world.

The agent did not know about the webhook test panel because the panel was added or surfaced after the cutoff, or because the documentation it was trained on did not emphasise it. It told me what made sense based on the world it had been trained on. The world had moved. I could see the world had moved, because I was looking at the dashboard. The agent could not, because the agent had no way to look unless I asked it to.

The pushback was the request to look. The third pushback was specific enough that the search actually surfaced the relevant documentation.

The harder lesson

The lesson is not "always push back." That is too broad and would just produce noise. The lesson is closer to this: when an agent gives you an answer that does not match something you can directly see, the friction you feel is signal. It is the most reliable signal you have that the agent is working from stale or incomplete information.

If you ignore the friction, you will spend the next two hours building toward a solution that was never necessary. If you trust the friction and push back, you will sometimes find out the agent was right and you were wrong, which is also useful. But the reliable failure mode is the one where the agent sounds confident, the answer is plausible, and you can see something that does not fit. That is when you push back.

This is the part most people are not doing. The disclaimer at the bottom of every AI chat interface says "always double-check." In practice, almost nobody does. The answer sounds right. It is delivered fast. The cost of accepting it feels lower than the cost of investigating. The pattern is the same one humans use when they read tweets, headlines, or anything authoritative-sounding that confirms what they already half-believe.

What I would tell a newer user

If you are six months in and you have not yet had the experience of pushing back on an agent and being right, you will have it soon. When it happens, remember the feeling. The thing that saved me from giving up on the webhook test was that I had been here before. Earlier versions of me had given up. Later versions of me knew that giving up was almost always the wrong move when something did not match what I could see.

You do not need to push back on every answer. You need to develop the sense for when the answer does not match what you can directly verify. That sense takes time. The way you build it is by pushing back the few times something feels off and watching what happens. Sometimes you were wrong. Sometimes the agent was wrong. Both outcomes calibrate the sense for next time.

The infrastructure I built for myself, which I have written about on this site, exists to handle a related but different failure mode: agents claiming a state about the world that is not true. Git state, project state, what files are open, what the current configuration is. That problem is solvable with infrastructure because the truth is local to my machine and I can put it in front of the agent on every session start.

The problem in this post is harder, because the truth was on Meta's servers, not mine. There is no infrastructure I can build that will reliably surface every Meta documentation update into my agent's context. The only available tool there is judgment. Specifically, the judgment to push back when the answer does not match what I can see.

That judgment is the most underrated skill in working with AI agents in 2026. It is rarely talked about because it does not sound technical. It sounds like "trust your gut." But it is not gut. It is a calibrated sense, built over months of working alongside these tools, that knows the shape of their failure modes and recognises one when it is happening.

The cost of not having it is real. People give up. They submit fifty-line products for review. They build around problems that never existed. They take the agent's first answer as final and walk away from work that would have been finished if they had stayed in the conversation for one more round.

Do not give up too early. When the answer does not match what you see, push back. The third round is often where the truth shows up.

---

This is part of a series about building products as a solo founder. Earlier posts cover why the Founder Knowledge Graph exists and the slogan that sells one part of the chain. More coming.