The offline promise is a test, not a sentence

The offline promise is a test, not a sentence

AEFRI engineering note · 2026-10-29

Most "works offline" claims in software are marketing copy. This one is a unit test that fails the build.

I ship a small pomodoro timer as a single 29,862-byte HTML file. There is no build step, no package manager, no server, and no CDN. You download the file, double-click it, and it runs. The file is 29 KB because there is nothing in it except the app.

That is only interesting if it stays true. The moment someone adds a font <link>, an analytics snippet, or a CDN-hosted library, the file still works on a machine with internet and quietly stops working on a plane. Nothing breaks. The build stays green. The download still says "no account, no internet" — and the claim rots.

So the guarantee is enforced in code:

def test_artifact_makes_no_network_calls(self):
    for probe in ("http://", "https://", "fetch(", "XMLHttpRequest",
                  "import(", "navigator.sendBeacon",
                  "<link rel=\"stylesheet\""):
        self.assertNotIn(probe, self.html,
                         f"offline promise broken by {probe}")

Seven probes. If any of them appears in the shipped file, the test fails, the release does not happen, and the person who added it gets a message naming the exact string that broke the contract. There is no flag to set, no environment variable to set for "we're only staging this".

The second test is the one that matters

A string check proves the file does not mention the network. It does not prove the app works. Those are different claims, and a lot of "offline first" software only makes the first one.

So there is a browser test:

page.goto(uri)                       # a file:// URL, not a server
page.wait_for_timeout(400)
self.assertEqual(page.inner_text("#clock"), "25:00")
page.click("#startBtn")
page.wait_for_timeout(2100)
self.assertNotEqual(page.inner_text("#clock"), "25:00")
page.fill("#taskInput", "Real task")
page.press("#taskInput", "Enter")
page.reload()
self.assertIn("Real task", page.inner_text("#taskList"))

The file is opened from disk with a file:// URL. If it needed a server it could not even start. The test then checks the timer counts down, a task can be added, and the task is still there after a reload — because the state lives in localStorage, and if that regressed the app would look fine for about five minutes before quietly losing everything.

The test also collects pageerror events and asserts the list is empty. An app that throws on load but renders a static shell would otherwise sail through.

Why the string test is still there

The browser test alone would not catch a font CDN, because Chromium in the test environment has internet access and would happily load the font from Google. The offline promise would hold in CI and break for every buyer who is actually offline.

Both tests together are what make the claim true: one proves the file does not depend on the network, the other proves it still works without one. Neither is sufficient alone, and it took me writing both to realise the browser test was quietly green for a week while the app pulled a webfont.

The previews are the artifact too

Another thing I did not expect: the product page previews are screenshots of the real file, not mockups. The renderer opens the shipped HTML in Chromium at the same dimensions and captures it, in both the dark and light theme. A mockup would have been easier and would have looked better — it would have been 200 KB of hand-tuned marketing art instead of a screenshot of something that might not exist.

The consequence is that the preview cannot lie. If the app breaks, the screenshot changes. If the app looks bad, the screenshot looks bad. Nobody has to trust the copy.

What an AI agent changes about this

The app itself was built by an AI agent working from a written specification, and I want to be precise about what that did and did not change, because the obvious framing ("AI writes code now") is the least useful part.

What it changed: throughput and the boring half of the work. A 29 KB single-file app with two themes, a task list, a notes pad, export/import, and keyboard shortcuts is maybe a day of careful work. It took an afternoon. The agent was also good at the parts I do not enjoy: writing the README, keeping the description in the listing copy in sync with what the app actually does, and generating the preview images.

What it did not change: the need for a falsifiable claim. An agent that asserts "this works offline" in the listing copy has accomplished exactly nothing. The value of the test is that it is the same kind of artifact whether a human or an agent wrote the code — it either passes or it does not, and it can be run by anyone.

There is a second lesson, which is about my own process rather than the agent's output. My first version of the offline test checked for http:// and https:// only. The agent then shipped a version that pulled a font from a CDN — not with a literal https:// in the source, but in a <link> tag I had not thought to probe for. The test did not fail because I had not written the probe, not because the agent was careless. The gap was in the specification, and the test suite surfaced it exactly as well as it would have surfaced a human mistake.

That is the whole argument for test-first specifications in agent workflows: the specification is where correctness lives, and the tests are how you find out whether you actually wrote it down.

The practical version

If you make an offline claim about your software, try to write the test that would falsify it before you write the sentence that makes it. Two tests are usually enough:

  1. A static check for network primitives in the shipped artifact, including

the ones you did not think of — stylesheet links, import(), beacons.

  1. A browser test that opens the artifact from disk, exercises the main

path, and asserts zero page errors.

If you cannot write the first one, that is a signal about the artifact, not about the test.

A note on disclosure: this article was drafted with AI assistance and edited for accuracy before publication. Every number quoted here comes from the build, and the tests described above run on every release.

If you want to see the thing itself rather than talk about it, the app and the rest of the small catalogue are at <https://aefristore.web.app>.