Key takeaways
- We push our AI agent prompts from files instead of pasting them into a screen, so the wording is version controlled and reviewable.
- We confirmed each push by reading a field the platform handed back. That field turned out to be a stale copy stored one level up from the thing we were changing.
- It returned the same pre push text for every agent on the account, so the check passed identically whether the push had worked or not.
- A prompt we knew was wrong stayed live for three weeks behind a green light nobody had reason to distrust.
- The rule that came out of it: verify by reading the exact field you wrote, or by watching the behaviour. Nothing else counts.
Our text and email assistant answers new leads. What it says is decided by a prompt, and that prompt lives in a file in our own repository, not in a text box inside the platform. A script pushes it up. That is on purpose. A prompt in a file can be reviewed, diffed, and rolled back. A prompt in a text box is whatever the last person typed.
The push script does the sensible thing at the end. It writes, then reads back, then tells you what it found. For weeks it told us everything was fine.
The field that always agreed with us
The platform exposes several fields for an agent. Two of them are the ones we actually write: the personality, which sets the voice, and the instructions, which set the job. There is also a third, a combined view of the whole prompt, and it is the obvious thing to read when you want to know what the agent is currently working from.
So that is what we read. It came back full of text that looked right, and we moved on.
It was not the agent's prompt. It was an artifact the platform assembles and stores at the account level, a level up from the individual agent. It does not refresh when you write to an agent. And because it sits above all of them, the same text came back for every agent on the account.
That is the part worth sitting with. The check was not unreliable. It was not flaky. It was a constant. It returned the same answer on a successful push, a failed push, and a push that never happened. A test that cannot fail is not a test.
Three weeks of a wrong prompt
The cost was real. We had identified a problem with one of the live prompts and fixed it in the file. The push ran. The readback looked correct. Everyone involved, including the automation writing the report, believed the corrected version was live.
It was not. Customers kept getting the old behaviour for three more weeks. Nothing in the system was flashing red, because from the system's point of view nothing had gone wrong. The write may well have succeeded on some of those runs. We genuinely cannot say, and that is its own lesson. A verification step that cannot distinguish between outcomes also destroys the record of which outcome you got.
What eventually caught it was a person noticing the agent still doing the thing the fix was supposed to stop. Behaviour caught it. The instrumentation never did.
The signal we walked past
There was a tell, and we had already seen it.
Four different agents, doing four different jobs, returned byte identical text from that field. At the time it read as a quirk. It should have read as an alarm. Identical output from things that are supposed to differ is almost never a coincidence. It usually means you are reading something that is upstream of the difference you care about, or a cache, or a default.
If you take one habit out of this post, take that one. When a check returns the same value for inputs that should produce different values, stop and find out why before you trust anything else it says.
What we do now
The rule is short. Read back the exact field the write targeted. If the write went to the instructions, read the instructions. Not the summary next to it, not the combined view, not the thing the interface displays.
Behind that sits a general principle about platform APIs, and it applies far beyond one product. Fields come in two kinds. There are the ones you own, which hold exactly what you put in them. Then there are derived fields, which the platform builds for its own display purposes out of other things. Derived fields can be cached, can be assembled on a schedule, and can be stored at a completely different level of the account than the thing you edited. They are useful for looking at. They are not evidence.
The second rule is that behaviour beats both. For an agent that means sending it a message and reading what comes back. That is slower than a script call and it is the only check that tests the whole path instead of one hop of it.
What this means for your business
Most small businesses now have something automated writing on their behalf. A text responder, an email follow up, a chat widget on the site. Almost nobody has a way to prove what it is currently saying.
Ask two questions about anything automated in your business. First, when somebody changes it, what exactly do they look at afterwards to confirm the change landed? Second, could that thing look identical if the change had silently failed?
If the answer to the second one is yes, you do not have a check. You have a habit that feels like one. The fix is usually not complicated. Send yourself a real message from the real system and read it. It takes a minute and it is the only test that cannot lie to you.
The wider version of this is worth saying plainly. Automation does not fail loudly very often. It mostly fails quietly, keeps reporting success, and costs you weeks. The green lights on a dashboard are only worth what the checks behind them are worth, and a check nobody has ever seen fail deserves suspicion rather than trust.
Want this built properly
We build and audit CRM automation for small businesses, and a good share of that work is finding the things that quietly stopped working while the dashboard stayed green. If you are looking for help with Go High Level in the Bay Area, see what we do at optechsol.llc.