Upgrade and Migration
How to move a running deployment to a new version, what can change underneath it, and how to get back if it goes wrong.
Version-specific steps belong with each release. This page is the procedure that applies to every upgrade and the principles behind it.
The one thing to understand first
A fresh install needs no migration. An existing install may need all of it.
On first boot the platform seeds its database from the files shipped with it — the configuration schema, the dashboard widgets, the standard policies, the menu structure, the algorithm rankings. That bootstrap writes only what is not already there, which is exactly right: it must never overwrite a customer's own configuration on restart.
The consequence is that a new setting shipped in a later version does not appear in a database that already has a configuration document. Restarting does not add it. A new dashboard widget does not appear. A renamed field is not renamed. None of this produces an error, because each of these is read by a query that simply returns nothing.
That is why an upgrade has a migration step, and why the migration step is not optional.
The failure mode to expect
Every migration this product has needed shares a shape worth recognising, because it is how the database tells you nothing is wrong when something is:
MongoDB queries a field that does not exist by returning no documents, not by raising an error. So a field renamed in code and not renamed in data produces an empty result that looks exactly like "there is none of that". A dashboard shows zero. A filter matches nothing. A tag disappears from every material. Nothing is logged, because from the database's point of view nothing failed.
This is why migrations here are run deliberately and verified afterwards, rather than attempted automatically at startup.
Before you start
| Step | Why |
|---|---|
| Read the release notes for every version between yours and the target | Migrations are cumulative. Skipping a version does not skip its migration. |
| Take a full backup of both databases | CBOMData and CBOMAudit. See Backup and Recovery. |
| Verify the backup restores | An untested backup is not a rollback plan. |
| Record the current version and configuration | You need to know what to go back to. |
| Note any hand-edited configuration file | See the warning below. |
| Schedule a window | Discovery should be quiet. Running agents will spool and republish. |
! A section hand-edited into a configuration file will not survive. The YAML files are generated from the database on every configuration save, activation and startup. The regeneration is wholesale — it writes exactly what the database says and nothing else, and nothing logs what was removed. If you have hand-edited a file, move that setting into the database before upgrading, or it will be gone afterwards and the loss will be silent.
The procedure
1. Back up both databases.
mongodump --uri "<connection string>" --db CBOMData --archive=cbomdata.gz --gzip
mongodump --uri "<connection string>" --db CBOMAudit --archive=cbomaudit.gz --gzip2. Save the current permission map. Authorisation is driven by a {role: [scope]} map in the database. Dumping it before an upgrade gives you a one-command undo if a new version's scopes land differently from how you had them.
3. Stop the application services. Leave the database, the broker and the identity provider running — the migration tools need the database, and the broker should keep holding anything agents publish during the window.
4. Take the new version and rebuild the images.
5. Run the migrations the release calls for, in the order it gives. Every migration tool follows the same contract: run it with no flags and it reports what it would change; run it with --apply and it makes the change. Always run the report first and read it.
python tools/<MigrationTool>.py # report only
python tools/<MigrationTool>.py --apply # make the changeRun them against each database the deployment has.
6. Start the services and watch the first startup through. A migration that did not run shows up here as a missing setting or an empty dashboard, not as an error.
7. Verify — see the checklist below.
What the migration tools do
They fall into four kinds. Knowing which kind you are running tells you what to check afterwards.
| Kind | What it changes | How a missed one shows up |
|---|---|---|
| Field rename | The spelling of a stored field, so code and data agree again | An empty result where there should be data — a dashboard reading zero, a tag that has vanished |
| Configuration top-up | Adds a section the seed would have added on a fresh install | A setting missing from System Configuration, or a service falling back to a default |
| Seed synchronisation | Brings database-held definitions up to the shipped set | A new widget or policy that does not appear |
| Repair | Corrects data a defect produced | Wrong figures that look plausible |
The field renames are the ones to take seriously, because their failure mode is invisible. A database that has not been migrated through a field rename loses the affected data on the next pass that rewrites it — not at the moment of the upgrade, but later, when something recomputes.
Verification checklist
Work down this list after every upgrade. Each line checks a different subsystem, and together they cover every migration kind above.
| Check | Confirms |
|---|---|
| Sign in with a local account and with SSO | Identity and session handling |
| Open System Configuration | The configuration schema and form agree |
| Open the main dashboard | Widgets resolved and the inventory reads |
| Open a material's detail and its history | CBOMAudit is reachable and intact |
| Check tags on a material you know is tagged | Any tag-related rename applied |
| Open Roles and Permissions | The permission map survived |
| Run a small discovery task end to end | Discovery, ingestion, deduplication and the dashboards |
| Check the deployment response's counters | Ingestion is accepting, not silently skipping |
| Open the audit trail | The audit plane is recording |
| Check the logs directory has current files | Logging paths resolved |
The discovery run is the one that matters most. It is the only check that exercises the whole path, and most migration problems surface as a scan that reports success and changes nothing.
Upgrading the components underneath
The database. Follow MongoDB's own upgrade path — one major version at a time, with the feature compatibility version raised only after the upgrade is confirmed. On a replica set, upgrade secondaries first and step the primary down last.
The identity provider. Upgrade separately from the platform, not in the same window. Existing sessions continue until they expire; new sign-ins fail while it is down.
The broker. Rolling, one node at a time, waiting for quorum queues to report synchronised. Agents spool while it is unavailable and republish afterwards, so a scan is delayed rather than lost.
Deployed agents. Agents are upgraded by redeploying the task that installed them. A deployed agent carries its broker address and credential in its own configuration, so a change to either means a redeployment — which is the main reason to settle those addresses before a large rollout rather than after.
Old agents keep working. The ingestion path keeps compatibility reads for agents deployed before a field rename, so an estate does not have to be upgraded in one go. Those reads are marked in the code and are removed once no old agent remains, so do not treat an indefinite mixed estate as supported.
Rolling back
Rolling back is a restore, not an un-migration. The tools apply changes; they do not reverse them.
Stop the application services.
Restore both databases from the backup taken in step 1.
Deploy the previous version's images.
Start, and work through the verification checklist.
Two things to know before you need them:
The permission map has its own restore path. If an upgrade left authorisation in a state that locks someone out — including an administrator — the saved map from step 2 can be written back without restoring the whole database.
Configuration has its own rollback. Configuration is versioned in the database, so a bad configuration change can be rolled back to a previous version from the console without touching the rest of the platform. That is a different operation from a version rollback and a much cheaper one.
Data discovered after the backup is lost on a rollback. Agents that spooled during the window will republish and their results will land again; anything already ingested and then rolled back has to be rediscovered. On a large estate, the cost of a rollback is mostly a rescan.
Troubleshooting
A setting is missing from System Configuration after an upgrade. A configuration top-up migration did not run. Run its report and read it before applying.
A new dashboard widget does not appear. The widget definitions were not synchronised. Widget documents are authoritative at runtime, so shipping the handler changes nothing visible until the document is there.
A dashboard reads zero where it read a number before. A field rename migration did not run. Stop and run it before anything recomputes.
Tags have disappeared from materials. The same cause, and the most urgent version of it. Restore from backup if a deduplication pass has already run — the pass rewrites the field, so the un-migrated data is gone rather than merely unreadable.
Nobody can sign in, including administrators. The permission map. Restore it from the dump taken before the upgrade.
The logs directory is empty. The logging handler paths did not resolve, so the subsystem fell back and created the directory without writing to it. There is a migration for this.
A scan reports success and the inventory does not change. Check the deployment response's skipped count, then the collector worker. A validator rejecting documents after an upgrade means the sensor and the schema disagree.