Switch storage backends

Estimated reading time: 3 minutes

Starting in DTR 2.6, switching storage backends initializes a new metadata store and erases your existing tags. This helps facilitate online garbage collection, which has been introduced in 2.5 as an experimental feature. In earlier versions, DTR would subsequently start a tagmigration job to rebuild tag metadata from the file layout in the image layer store. This job has been discontinued for DTR 2.5.x (with garbage collection) and DTR 2.6, as your storage backend could get out of sync with your DTR metadata, like your manifests and existing repositories. As best practice, DTR storage backends and metadata should always be moved, backed up, and restored together.

DTR 2.6.4 and above

In DTR 2.6.4, a new flag, --storage-migrated, has been added to docker/dtr reconfigure which lets you indicate the migration status of your storage data during a reconfigure. If you are not worried about losing your existing tags, you can skip the recommended steps below and perform a reconfigure.

Best practice for data migration

Docker recommends the following steps for your storage backend and metadata migration:

  1. Disable garbage collection by selecting “Never” under System > Garbage Collection, so blobs referenced in the backup that you create continue to exist. See Garbage collection for more details. Make sure to keep it disabled while you’re performing the metadata backup and migrating your storage data.

  2. Back up your existing metadata. See docker/dtr backup for CLI command description and options.

  3. Migrate the contents of your current storage backend to the new one you are switching to. For example, upload your current storage data to your new NFS server.

  4. Restore DTR from your backup and specify your new storage backend. See docker/dtr destroy and docker/dtr restore for CLI command descriptions and options.

  5. With DTR restored from your backup and your storage data migrated to your new backend, garbage collect any dangling blobs using the following API request:

     ```bash
 curl -u <username>:$TOKEN -X POST "https://<dtr-url>/api/v0/jobs" -H "accept: application/json" -H "content-type: application/json" -d "{ \"action": \"onlinegc_blobs\" }"
 ``` 
 On success, you should get a `202 Accepted` response with a job `id` and other related details.

This ensures any blobs which are not referenced in your previously created backup get destroyed.

Alternative option for data migration

  • If you have a long maintenance window, you can skip some steps from above and do the following:

    1. Put DTR in “read-only” mode using the following API request:

       curl -u <username>:$TOKEN -X POST "https://<dtr-url>/api/v0/meta/settings" -H "accept: application/json" -H "content-type: application/json" -d "{ \"readOnlyRegistry\": true }"

      On success, you should get a 202 Accepted response.

    2. Migrate the contents of your current storage backend to the new one you are switching to. For example, upload your current storage data to your new NFS server.

    3. Reconfigure DTR while specifying the --storage-migrated flag to preserve your existing tags.

DTR 2.6.0-2.6.4 and DTR 2.5 (with experimental garbage collection)

Make sure to perform a backup before you change your storage backend when running DTR 2.5 (with online garbage collection) and 2.6.0-2.6.3. If you encounter an issue with lost tags, refer to the following resources:

Upgrade to DTR 2.6.4 and follow best practice for data migration to avoid the wiped tags issue when moving from one NFS serverto another.

Where to go next

Rate this page:

 
0
 
0