0033 added the column and left the fill to the Jetstream backfill. That is fine
for a running system and wrong for replay: a database several releases behind
runs every pending migration back-to-back at boot, long before any worker
starts. Anything built on top of manifest_key would, on that path, silently
operate on NULLs while working perfectly on a system that had been up for a
while. Establishing a migration's data precondition out-of-band means replay
cannot see it.
The runner now supports a Go step per migration version, running inside the same
transaction as that migration's SQL, after the DDL it depends on and before the
version is recorded. A version is never recorded without its Go half.
0033's step fills manifest_key for every row lacking one. It has to be Go: the
value is a truncated sha256 and SQLite has no hash builtin. It pages through the
table and writes one UPDATE ... CASE per 500 rows, because a statement per row
would be correct and unusably slow against a remote primary.
Verified by unregistering the hook: replay then leaves 3 of 3 seeded manifests
with NULL keys. With it, all three are filled, from a snapshot of the pre-0009
schema forward.
The upserts keep their "OR manifests.manifest_key IS NULL" clause. It is a
self-healing net for rows that somehow arrive without a key rather than the
mechanism anything depends on.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Migrations can be recorded without being executed. That is not hypothetical:
migration 0009 exists to clean up after 0004, which production recorded but
never applied, leaving eleven columns behind that fresh installs never had.
Nothing reported it at the time; it surfaced later as confusing behavior.
InitDB now compares an existing database against schema.sql after migrations run
and logs one warning per difference. Fresh databases skip the check, since they
were just built from schema.sql and agree by construction.
The comparison works by applying schema.sql to a throwaway in-memory database
and introspecting that, rather than parsing the DDL. SQLite's own resolution of
types, defaults and implicit indexes is exactly what we want to compare against,
and a hand-rolled parser would drift from the engine. The introspection is
shared with TestSchemaMatchesMigrations, so the test exercises the same code
that runs at boot.
Warn-only, never fatal. A database merely ahead of or behind schema.sql is
almost always still able to serve traffic, so refusing to boot would turn a diff
that wants a corrective migration into an outage, during a deploy, which is the
worst possible moment to have one.
The README claimed new tables go in schema.sql only. That is wrong in the
direction that hurts: InitDB skips schema.sql entirely once schema_migrations
has rows, so such a table appears on fresh installs, passes every test, and is
silently absent in production. Documented the real rule along with two others
the test cannot enforce: migrations must not return rows (go-libsql rejects them
with "Execute returned rows"), and rebuild migrations must name columns
explicitly, since column order legitimately differs between fresh and upgraded
databases.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Evan JarrettandClaude Opus 5