Storage reorganisation · outstanding work

What Is Still Missing

The actionable remainder, ordered by what blocks what. Every figure was read out of the code or the live databases on 2026-09-08 — this is a punch list, not an estimate.

Steps outstanding: 4 of 6Two partial, one unrun, one unbuilt
Gates passed: 0 of 3Gate 3 never attempted
Columns uncovered: 2655,137 values with no descriptor
Bare references: 1,1221,005 of them unfixable today
Prod migrations behind: 4capClone cannot run the copier

The one-paragraph version

The data conversion is finished and the thumbnail service is live. What is missing is everything that touches the objects: the copier has never been run, it has no descriptor for 26 of the 73 columns that can hold a path, production is four migrations short of being able to run it at all, and the two steps that make the old folders retirable — the upload shape flag and legacy-read logging — are unbuilt.

Blocking, in order

Each of these gates the next. Nothing below the copier can be attempted until the copier has run once.

Bring capClone up to date

not started

What is missing: Four migrations applied in clone1 are not in production: 20260902101440 (PO paths), 20260905110855 and 20260905110946 (the two ledger tables and the run switch), and 20260908064726 (the ledger refs to ref_data jsonb).
Why it is first: The copier works through file_migration_queue and file_migration_batches and is started by file_migration_enabled. None of the three exists in capClone, so there is nothing to run and nothing to run it with.
Also: transactions.poDocument in production has not had the offline purchase-order prefix applied that clone1 has.

Run the rehearsal and close Gate 3

not started

What is missing: One account copied, verified, and rolled back once for real. The queue has never been seeded — both ledger tables hold zero rows in clone1.
Why it matters more than the rest: Every later batch depends on the rollback working, and a rollback that has never been run is a claim rather than a fact.
The cheap way to do it: Set FILE_MIGRATION_TARGET_PREFIX. Every destination is then written under a scratch folder, and because the prefix is baked into destination_path at seed time rather than applied at copy time, the write-back and the rollback are exercised against the real database. That is a genuine end-to-end test that touches no real destination.

Decide the 26 uncovered columns

not started

What is missing: PATH_COLUMNS holds 73 entries and MIGRATION_DESCRIPTORS reaches 13. Two more are excluded by decision. Of the rest, 26 hold real file paths — 55,137 values across 53 key paths — and no run will copy their objects or rewrite their rows. Enumerated in full on CAP-680 Gaps.
Start with: biomeds.extracted_data (20,052 references to 159 distinct objects, so only 162 copies are owed) and camp_inventory_items.additional_details (15,319 under cad.cadUrl). Neither table is named anywhere in the plan.
The one that needs a decision first: budget_planning.attachments is a confirmed file-path column, so its 161 bare values are real references — they carry the CapExpert naming convention (1670237033872_Z4USQP6VGO.jpg) and resolve against attachments/budget-planning. That folder exists as a BucketFolder but has no entry in UPLOAD_MODULE_TARGETS, so unlike every other uncovered column a descriptor cannot simply be added: a module has to be defined first — tree, scope and domain — because the destination is derived from the module through finalPrefixFor and there is nothing to derive it from. It is also still sitting in the name alone half of PATH_COLUMNS, so that comment is now out of date and worth moving up with the traced writers.

Defects to fix before a real run

What Scale Consequence

Destination comes from the descriptor's module, not from where the object sits equipment_models.files 5,088 of 5,088; transactions.poDocument 269 of 738; transactions.buyerInvoice 656 of 961 Files land in a domain their kind does not belong to. Nothing is lost — the record is rewritten and still reads — but the per-account tree is the access model.

Multiply-prefixed values in copilot_requests.proposal_summary 266 rows doubled, 174 of those tripled; 265 distinct values The paths point at objects that do not exist. The "prefixed twice" root cause the audit fixed for five sites, still accumulating in an uncovered column.

1,005 bare file names in columns with no descriptor 823 of them in camp_inventory_items.additional_details across six keys Worse than an uncovered complete path: a bare name needs a reader that already knows the folder, which is the condition the migration exists to end.

38 array entries pointing at deleted stickers camp_inventory_items.images, identical in capClone Pre-existing. Cleanup, not a blocker.

What	Scale	Consequence
Destination comes from the descriptor's module, not from where the object sits	`equipment_models.files` 5,088 of 5,088; `transactions.poDocument` 269 of 738; `transactions.buyerInvoice` 656 of 961	Files land in a domain their kind does not belong to. Nothing is lost — the record is rewritten and still reads — but the per-account tree is the access model.
Multiply-prefixed values in `copilot_requests.proposal_summary`	266 rows doubled, 174 of those tripled; 265 distinct values	The paths point at objects that do not exist. The "prefixed twice" root cause the audit fixed for five sites, still accumulating in an uncovered column.
1,005 bare file names in columns with no descriptor	823 of them in `camp_inventory_items.additional_details` across six keys	Worse than an uncovered complete path: a bare name needs a reader that already knows the folder, which is the condition the migration exists to end.
38 array entries pointing at deleted stickers	`camp_inventory_items.images`, identical in `capClone`	Pre-existing. Cleanup, not a blocker.

Genuinely unbuilt

Everything above is repair or execution. These two are new construction, and they are what makes the legacy folders retirable.

Step 4

The upload shape flag

not started

What is missing: A setting that gates the upload shape. finalPrefixFor returns the new prefix unconditionally. file_migration_enabled is a different flag — it is the copier's run switch.
Consequence: Step 4 is live in effect and can only be reversed by a deploy. The ordering worked out — step 3 is deployed, so those uploads did get their thumbnails — but by circumstance rather than by design.

Step 6

Legacy-read logging by app version

not started

What is missing: Nothing records who still reads the legacy folders. The first half of step 6 is true by construction — finalPrefixFor only produces new-layout paths, so nothing writes to them any more.
Consequence: The plan says evidence, not a date, decides when the old folders can go. Without the log there is no evidence to decide on, and the folders stay indefinitely.

Step 2

The mobile leg

not started

What is missing: Neither mobileApp nor mobileApi carries the shared path rule. mobileApi has CAP-2031 commits, but they are unrelated release fixes on release/v15.
Consequence: It holds Gate 1 open, and it is the half of Gate 2 that cannot be tested — "an upload from web and phone lands correctly".

The safeguard that would have prevented most of this

Two lists that already exist, never compared

PATH_COLUMNS is the authoritative list of every column an upload path can reach — the temp sweep's validate() refuses to delete anything until every entry resolves against the live schema. MIGRATION_DESCRIPTORS is the list of columns the copier handles. Nothing compares them.

A test that walks PATH_COLUMNS and fails on any entry with neither a descriptor nor a recorded exemption would have caught all 26 gaps at authoring time. A second assertion — that no column is left holding a bare name once its descriptor has run, with a declared exemption for a label key that has a reference sibling — would have caught the 1,122 bare references.

Both are cheap. The asymmetry is already understood in the codebase: path-references.ts argues that a column listed but empty costs one sequential scan a day, while one missing from the list can license the delete of a file still in use. For the copier the same asymmetry runs the other way, and nothing enforces it.

Not missing

Recorded so the remainder is not read as bigger than it is.

Item State

Complete paths in every file column and JSON key 1,488,594 of 1,488,632 values

One shared path rule in the API and the web app upload-targets.ts, mirrored in the front end

Staging, filing and the daily reclaim of abandoned uploads Built and running

The thumbnail service Deployed as a Cloud Function; backfill run; 78 of 78 tests pass

The copier itself — seed, run, write-back, rollback, console Built and reviewed; only unrun

Legacy mirroring for pre-img/ readers LEGACY_MIRRORS in the thumbnail service

Thumbnail backfill for existing images Done

The two skipped descriptors ezestimator_requests and capture_billings are empty tables

Item	State
Complete paths in every file column and JSON key	1,488,594 of 1,488,632 values
One shared path rule in the API and the web app	`upload-targets.ts`, mirrored in the front end
Staging, filing and the daily reclaim of abandoned uploads	Built and running
The thumbnail service	Deployed as a Cloud Function; backfill run; 78 of 78 tests pass
The copier itself — seed, run, write-back, rollback, console	Built and reviewed; only unrun
Legacy mirroring for pre-`img/` readers	`LEGACY_MIRRORS` in the thumbnail service
Thumbnail backfill for existing images	Done
The two skipped descriptors	`ezestimator_requests` and `capture_billings` are empty tables