Key takeaways
- Our own written note said a security setting was capped by the free plan and that upgrading would lift it. So the account upgraded, the number went up, and logins started returning server errors.
- The cap is not a plan feature. It is a deliberate refusal in the platform's runtime, documented in their own public issue tracker, and it applies on every plan. Paying does not move it.
- Only failed logins broke. Correct passwords kept working, which is why nobody saw it immediately and why the reports made no sense when they arrived.
- An internal note that explains why something is limited is a claim. A claim nobody has tested is a guess with a confident voice.
- A security parameter is the worst place in your system to raise a number on trust, because the failure is total and it lands on everybody at once.
Passwords in the client portals we run are not stored. They are put through a slow, deliberately expensive calculation, and only the result is kept. How slow is a number in the code, and the higher that number the more expensive it becomes for anybody who steals the database to guess their way in.
Ours sat at eighty thousand. The published guidance for that algorithm asks for six hundred thousand. A note in our own documentation explained the gap: the free plan caps how much processing a single request may use, and upgrading would lift the cap.
So we upgraded, and moved the number to two hundred and ten thousand.
Every wrong password started returning an error
Not every login. That is the detail that turned a ten minute fix into a long afternoon.
Anyone typing the correct password sailed through. Anyone typing the wrong one, or an address that did not exist, got a server error instead of the polite try again. Rolled back, everything returned to normal, and we were left holding a symptom that appeared to contradict itself.
The reason is a small piece of deliberate design. When a login misses, the system still runs the expensive calculation before saying no. It runs it so a wrong email address and a wrong password take the same amount of time, because a login that answers instantly for unknown addresses tells an attacker which of your customers have accounts.
So the expensive step was on the miss path. Raising its cost broke the miss path. Successful logins never touched the broken part.
The number that killed our theory
We went in believing the processing limit explanation, because we had written it down ourselves, and a written note reads like a finding rather than an opinion.
If a request is being killed for using too much processing, the broken version should be slower than the working one. It should run, chew through its budget, and get cut off at the ceiling.
The broken version was faster. A hundred and sixty one milliseconds, against about two hundred and fifty for the healthy build.
It was faster because it was not doing the work at all. The runtime looked at the number, refused, and threw immediately. That single measurement is the whole diagnosis, and it points the opposite way to the theory we walked in with.
A constant, not a budget
Once we stopped trusting our own note, the bisect was quick. A hundred thousand worked five times out of five. A hundred and fifteen thousand failed five times out of five. No middle, no flakiness, no dependence on load or time of day.
Budgets are fuzzy. They fail sometimes, under load, on the slow request and not the fast one. This was a wall. Which means it is a constant somebody typed, and it turned out to be exactly that: a guard written into the runtime on purpose to stop a single request burning the platform's processors, sitting at exactly one hundred thousand, identical on every plan.
The practical consequence is almost funny. Our eighty thousand was margin, held back from a ceiling we thought might move under load. The ceiling does not move at all. A value at or under it always works and a value above it always fails, so there was never any reason to leave room. The portal now runs at exactly one hundred thousand, and there is a test that asserts the number so nobody raises it again on a Friday.
That test is the interesting part, because a normal test cannot catch this. The same code running on a developer machine will happily do two hundred and ten thousand. The refusal exists only in the deployed runtime. So the test does not try the calculation. It reads the constant and fails if it is wrong.
Three things that made this expensive
First, the live log tool wedged and produced nothing, as it had during an earlier investigation months before. The diagnostic that worked was a plain bisect against the deployed site, changing one number and watching the response.
Second, deploys take a moment to settle, and requests during that window can fail either way. Our first rollback looked like it had changed nothing. Give any deploy about forty five seconds before you judge it, including the one you are using to undo a mistake.
Third, and the one worth carrying: the corrected note in our documentation now says what was wrong with the old one and what it cost. An internal note that quietly gets overwritten teaches nobody anything.
What this means for your business
You have notes like ours. Every business does. Somebody wrote down why a thing is set the way it is, and that explanation has been quoted ever since without anybody going back to check whether it was ever true.
The ones that will hurt you are the ones that end in a recommendation. It is limited because of X, so if we ever do Y we can change it. That sentence is doing two jobs and only one of them was ever tested.
Before you act on a note like that, ask what evidence sits behind the because. If the answer is that somebody reasoned it out at the time, treat it as a hypothesis and test it in something small before you touch the thing everybody logs into. Any Go High Level Bay Area operator running real systems will tell you the same thing: the settings that take you down are the ones you changed confidently.
Want this built for you
We build systems where the security settings are pinned and explained, plus the websites and funnels feeding them. Start at optechsol.llc.