Migration
About in-place upgrade
NAbox 3 cannot be upgraded in place to NAbox 4, this is due to a revamped architecture and a change of the base OS.
NAbox 3 was using Alpine linux as a foundation, while NAbox 4 is using Flatcar Container Linux.
Pre-migration checks
- Verify NAbox version
You need NAbox 3.5.2 minimum to be able to migrate data to NAbox 4
- Verify DNS settings
When systems are added with short host names, it's important that NAbox 4 has the same domain names configured as NAbox 3 to be able to resolve system DNS names
- Root certificates
If an enterprise CA is configured on NAbox 3, it is used to establish secured TLS connections to the NetApp systems. NAbox 4 needs to have the root certificates as well
- Disk size
Migrated data should take less space on NAbox 4 than NAbox 3, but you still need to grow the data disk size if it has been done with NAbox 3 in the past to make sure that the migration has enough capacity at the destination
- Virtual machine resources
If you increased NAbox 3 memory and/or CPU, you should check that NAbox 4 has the same resources configured.
Running the migration
- Connect to NAbox 3 instance with SSH as root, and install
migrate
CLI utility
# curl -o /usr/bin/migrate -k https://<NAbox 4 ip>/migrate && chmod 755 /usr/bin/migrate
# screen -R
# migrate
screen
session if that's not the case.
- You will be prompted for NAbox 4 IP address and
admin
password - The tool will go through the following steps :
- Verify and exclude already configured systems
- Confirm which systems should be configured (all by defaut)
- Add your systems to NAbox 4
- Enable Prometheus admin API
- Take a snapshot of Prometheus data
- Disable Prometheus admin API
- Copy Prometheus data to NAbox 4
Example session :
~ # migrate
┃ NAbox 4 IP Address
┃ > 10.3.0.157
NAbox 4 Password
> ******
┃ The folowing systems are already at the destination and have been ignored :
┃ - cluster2
┃
┃ Select systems to configure in NAbox 4 :
┃ > ✓ Webscale Demo
Restarting Prometheus with admin API enabled... OK
Waiting for Prometheus.. OK
Taking snapshot... OK
Restarting Prometheus with admin API disabled... OK
Waiting for Prometheus.. OK
Prometheus import mode
Prometheus snapshot stats:
blocks found: 14;
blocks skipped by time filter: 0;
min time: 1700636426825 (2023-11-22T07:00:26Z);
max time: 1714855433776 (2024-05-04T20:43:53Z);
samples: 3628977545;
series: 310991.
VM worker 0:↓ 375376 samples/s
VM worker 1:↓ 350265 samples/s
Processing blocks: 0 / 14 [▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒] 0.00%
The data migration can take several hours, this is why it is strongly
recommended to run it in a screen
session, so if the terminal session is
interrupted, the migration will continue.
If your session is interrupted, you can restore it by running screen -R
Manual Migration
If the migration tool doesn't work for some reason, the manual steps are described below. Note that it will only migrate Prometheus data and not import the systems configuration to NAbox 4.
- Enable guest access to metrics in NAbox 4
- Restart Prometheus with admin API
$ cd /usr/local/nabox/docker-compose
$ WEB_ENABLE_ADMIN_API=--web.enable-admin-api docker compose --env-file .env --env-file .env.custom up -d prometheus
- Make a Prometheus data snapshot using API
$ curl -k -X POST https://localhost/prometheus/api/v1/admin/tsdb/snapshot
{"status":"success","data":{"name":"20240404T061544Z-00b8acf6e99be9be"}}
$ export SNAPSHOT_PATH=/prometheus/data/snapshots/20240404T061544Z-00b8acf6e99be9be
Set the SNAPSHOT_PATH
environment variable with the path to the created snapshot
- Remove admin API from Prometheus
$ docker compose --env-file .env --env-file .env.custom up -d prometheus
- Migrate prometheus data to Victoria Metrics
$ NABOX_IP=<nabox_ip>
$ docker run --rm -v /data/prometheus:/prometheus -it --network docker-compose_default victoriametrics/vmctl:heads-master-0-g93a29fce4e prometheus --prom-snapshot=$SNAPSHOT_PATH --vm-addr=https://$NABOX_IP/vm/ --vm-insecure-skip-verify -s
- Disable guest access to metrics in NAbox 4
Revert the change in step 1