What we’ve tested, and what we haven’t
Most companies show you a list of security features. Here’s ours with an honest third column instead — proven means we’ve actually done it to the running system and watched it work, working means the code is live but we haven’t tried hard to break it yet, and owed means it’s meant to behave that way and we still owe you a real test.
We have not had an outside security audit. This list is the standard we hold ourselves to in the meantime.
| The test | What should happen | Status |
|---|---|---|
| Someone tampers with a pending settings change | Refused and quarantined. Nothing is written. | Proven |
| An update fails partway through | It checks its backup and undoes itself. | Proven |
| The web console tries to edit protected settings directly | Refused. Only one narrow, separate process can. | Proven |
| Someone asks the AI for a stored password | The file is refused before it’s opened. | Proven — re-ran every real read the system had performed against the rule; one refusal, and it was the right one |
| The AI tries to open a file type that holds keys or certificates | Refused, whatever the file happens to be called. | Proven — found by breaking it deliberately and watching the check catch it |
| The AI tries to change something important | Refused, and told where to put it instead. | Working |
| The AI tries to use a tool it wasn’t given | It can’t see it and can’t call it. | Working |
| The logging breaks mid-job | Your work carries on; the failure is recorded. | Working |
| A booby-trapped document tries to hijack the AI | No password available, no new abilities, nothing changes. | Working — we still owe a proper attack test |
| One installation tries to reach another’s files | Refused at the boundary. | Working — we still owe a proper attack test |
Two of those rows moved up in August, and it’s worth saying how. Not by writing new code and declaring it done — by deliberately removing the protection, watching the check catch its absence, and putting it back. A protection nobody has watched fire is a protection nobody has tested.
The failure we design around
The thing that has cost us most isn’t a break-in. It’s something quietly stopping and reporting success.
It always looks the same. A job fails but exits as though it worked. An error goes into a file nobody ever reads. A summary silently drops half its content and hands back something that looks fine. A safety check gets written correctly and then never actually runs.
In every case the dashboard stays green while nothing is happening. And green is more dangerous than red, because red gets investigated.
The blocklist above was exactly this. It ran for months, reported clean every time, and was clean only because it couldn’t see the files it was meant to be looking at. We found that by running two of our own checks against each other instead of assuming they agreed.
So these are hard rules here, not good intentions:
- Nothing reports success without accounting for what it was given.
- If something got cut short, that’s an error — not a result.
- A safety check isn’t finished until we’ve watched it actually trigger.
- Failing quietly to be helpful is a disguise, not a kindness.
- A protection that’s never run in the situation it’s meant for isn’t a protection.
- A scheduled job that can’t do its work has to fail loudly and then check its own result. “It ran” is not “it worked.”
- Couldn’t-check is not the same as clean, and must never be reported as clean.