Our help page said a feature did not exist. It had been working for eleven days.

You cannot tell which sentences in an app's help pages are still true

You are deciding whether some gardening app will actually remind you to bring the seedlings in. You read its help page. It says the feature is coming, or that something is not supported yet, or that a thing has not been tested.

You have no way to know when that sentence was last true. There is no date on it. Nothing about a documentation page tells you whether it describes the software you just downloaded or the software from eight months ago, and the sentences most likely to be stale are precisely the ones you are relying on — the ones about limits.

We are as exposed to this as anybody, and last week we proved it three times in a single day.

Eleven days of telling people something we had already done

Our hardening-off page carried this: no notification has yet been proven to arrive on a real device. It was an honest sentence when it was written. We would rather say a thing is unproven than imply it works.

It had stopped being true eleven days earlier. An alert had arrived on a real phone, with the app closed, and a note recording exactly that had been sitting in our own code since the end of July. A false sentence, on a live page, read by real people, contradicted by a file in the same repository.

Nobody was careless. There was simply no moment at which anyone was going to walk past that sentence again.

Then the same day, a page denying a feature another page describes

Our photos page said the app could not look at a picture and tell you what is wrong with a plant. It can. Another of our help pages documents it working, in detail.

So two of our own pages disagreed with each other, and the one that was wrong was the one telling you a shipped feature did not exist. If you had read that page and closed the tab, you would have been wrong about the product because of us.

A third turned up as well: a page describing a fix about an hour before the fix existed. Wrong in the other direction, and just as much our fault.

Why our existing safety net could not see any of it

We already had a check for this shape. It fires when the app changes and no help page changes with it — the we shipped something and forgot to write it down case. It is a good rule and it is completely blind here.

Because nothing changed. No commit, no deploy, no diff. The world moved: a notification arrived on a phone in somebody's kitchen. A check that watches for changes cannot see a sentence that expires quietly on its own.

That is the actual lesson, and it generalises past us. Some claims are made false by us, and some are made false by everybody else. Only the first kind leaves a trace.

What we built, and the part that did the work

Every sentence on every help page that says not yet, never, or no X so far is now classified, once, by a person. There are 33 of them across 22 pages.

Twenty are permanent — only we can make them false. We will never name a chemical dose. Nothing you write is ever sent anywhere. Those are policies and guarantees, and changing one is an edit, which our existing checks already see.

Thirteen are perishable — somebody else or the world can make them false. Each of those carries two extra things: the one sentence that would prove it wrong, and the date a person last looked. Go past the recheck window and the build fails.

And the part that actually caught things was not the check running. It was writing that falsifying sentence. Having to say out loud this becomes untrue the moment X happens is what made somebody notice that X had already happened. The photos error was found that way — not by any code executing, but by a person being made to finish a sentence.

Where we are still wrong

It covers help pages and nothing else. This post is not covered by it, and does not need to be: a blog entry is stamped with a date, so not yet in a piece dated 11 August is a statement about 11 August and cannot rot. An evergreen help page makes no such promise, which is exactly why it is the surface that needed this.

The classification is a human judgment, every time. No pattern can tell a policy from a prediction — both read as we do not do X — so a new one has to be looked at by a person, and the check's only job is to refuse to let a new one through unlooked-at.

And it caught its own author twice within ten minutes, on the correction to the very page that started all this. We are not reporting a solved problem. We are reporting a problem that now has to announce itself.

The thirteen are worth reading on purpose

Taken together, those thirteen perishable claims are an honest list of what this app does not do yet. Not a roadmap — a roadmap is a promise. A list of specific sentences, each with the exact thing that would make it obsolete, and a date somebody last confirmed it was still true.

That is a slightly uncomfortable document to keep in public, which is a fair sign it is the right one.