The Things I Set Once and Never Checked Again

Every new account in my journaling app sees a short screen called "Before you begin" before the first conversation. It explains what the app does with what you write. I wrote it in April, I have reread it a dozen times since, and this week I found it was sending people to the wrong place for some things. Rereading it was never the same as checking it against the app.

Some of it may never have been quite right, and some of it the app simply outgrew. Either way, the screen no longer matched the app. Over two days I found the same pattern in two settings, and it matched a mistake I had already paid for in September. I think the pattern matters more than any one of the fixes.

A screen that described an older app

I had opened the work expecting to reorder the screen, because an old note said safety should come first. Safety had been first since April. The note was the stale thing, and reading every version of the screen against the live app is what showed it.

That same read found the real problems. The last section said you could delete everything from Settings. Settings deletes the whole account, and a single conversation or entry is deleted from History, so anyone who wanted to remove one night's writing was being sent to the wrong place. The line about usage data was narrower than what the app actually collects, since it never mentioned error reports. And for a screen about an app people open at night when they might be struggling, it never said where to get help right away.

The new version says where each delete lives, names error data next to usage data, and puts the 988 crisis line on the screen itself.

Who should see it was the harder call. Showing it only to new accounts was the easy path. I went the other way, and every existing account sees the screen once more. That costs a little friction for people who already said yes. But they said yes to a description of an earlier version, and I do not think that yes should stretch to the version they are using now.

A setting with no end date

In late April I loosened one setting so I could test on preview builds, the throwaway copies of the app that get made for every change before it goes live. It was meant to last as long as the test did.

It lasted five months.

When I looked at it again this week, it was wider than anything I would sign off on today, so it is gone, and preview testing now runs inside a boundary I set on purpose. What bothers me is how it survived, because nothing about it was hidden. It sat in plain view in a settings page I had opened several times since, and each time it read as part of the furniture.

The same review turned up a second setting I had marked "recommended to tighten" in April and never scheduled. That one tightens the next time each piece of the server changes, and that end is now written down.

Giving everything a date

The third case is that September mistake. In April I summarized the new state laws on AI companions from law-firm write-ups and a tracker, and one of those summaries got a Washington requirement wrong. The error reached the product and a published case study before a full read of the law caught it in September. I have corrected both.

So the fix for all three is boring, and I have put it in place anyway. The outside facts I know the app leans on (a law, a vendor's terms, a standard) now sit in one table with where it came from, whether I read the source or a summary of it, the day I last checked it and the day I have to check it again. There are 48 of them. A second table holds 28 settings, each with the value it should have, the value it had when someone last looked, the day to look again, and, for anything loosened on purpose, the day or event that ends it. The first pass found five settings with no record of anyone checking them since they were set, and a security checklist I cite that has a new edition out.

I also changed one habit. When I write about a law or a vendor's terms, I reread the source that day. The table tells me where the source is. It does not tell me the source still says the same thing.

I notice, writing this, that I am doing something I have warned myself about. Two new tables of things to check is exactly the kind of setup I can mistake for progress. What keeps them honest, I think, is that each row ends in a date or a named event, the dates show up when I open my work each morning, and a check that is due gets done before anything relies on it. If they turn into something I maintain instead of something that makes me look, they should go.

Mental Models

A statement is only as good as its last check against the source. A disclosure, a setting or a summary is only as current as the last time someone checked it against the thing itself. Give it a date to be checked again, or treat it as unverified.

People should see the description of the version they are using. When what an app does changes in a way a user would care about, the people who agreed to the old description should see the new one.

A temporary exception needs its end written down when it is made. "Until the test is done," kept in your head, is not an end. Write down the day or the named event, or the exception becomes the setting.