Migrating from Gravity to K3s

Due to the suspension of support for Gravity by Teleport, Anaconda highly recommends that Gravity users migrate to a fully supported, up-to-date Kubernetes implementation. To aid you in the process and make it as painless as possible, Anaconda has developed a migration process that allows you to leave your data in place, meaning your cluster will retain its configurations, projects, deployments, and scheduled jobs.

​

Prerequisites

Before starting the Gravity to K3s migration process, ensure your system meets the following requirements:

• Verify that you have a functioning Gravity cluster running Workbench.
• Verify that you have 300GB disk space available in var/lib/rancher.
• Verify that you have SSH access from the control plane node to all agent nodes.

• Verify that you have the K3s directory bundle and the image tarball for the latest version of Workbench.
• Verify that Docker or Podman is removed from any VMs where K3s will be installed.
• Verify that your cluster’s SSL certificates are up to date.
• If you have created an anaconda-persistence volume, you need the .yaml file you wrote containing the PersistentVolume and PersistentVolumeClaim values. If you don’t have it anymore, recreate the file before you begin the migration process.
• Review the K3s installation requirements for your operating system and verify that your virtual machines are prepared to receive K3s:
  • SELinux must be disabled.
  • Anaconda strongly recommends that built-in firewall software is fully disabled during migration. If this is not possible, ensure that you open the ports listed here.
  • On RHEL/CentOS, the NetworkManager service must be disabled:
    systemctl disable nm-cloud-setup.service nm-cloud-setup.timer
    
  If Gravity is already successfully running, it is likely that your system already meets these requirements. Even so, Anaconda strongly encourages you to verify the requirements anyway, especially on RHEL.

​

Pre-installation

Perform all actions from an administrative account that has sudo access on all nodes and a non-zero UID:

1. SSH onto the primary node.
2. Deliver the latest Workbench image tarball and K3s bundle to the primary node.
3. Unpack the K3s bundle and cd into the directory that it creates.
4. Remove old installations of conda by running the following command:
   rm -rf ~/ae5-conda ~/miniconda3
   
5. Install and activate the ae5_conda environment by running the following commands:
   bash ae5-conda-latest-Linux-x86_64.sh
   source ~/ae5-conda/bin/activate
   
   This environment contains all the tools you need to manage Workbench; for more information, see Administration server.
6. Obtain your current cluster configurations by running the following command:
   ./extract_config.sh
   
   The extract_config.sh script produces two files: helm_values.yaml and anaconda-enterprise-certs.json.
7. Back up your current instance by running the following command:
   ae_backup.sh
   
   The ae_backup.sh script produces two timestamped files: ae5_config_db_<YYYYMMDDHHMM>.tar.gz and ae5_data_<YYYYMMDDHHMM>.tar.gz.
8. Stage the K3s assets by running the following command:
   ./k3s_preinstall.sh
   
   The k3s_preinstall.sh script produces two files: k3s_values.yaml and NEXT_STEPS.txt. The preinstallation script is designed to coach you on the necessary actions you need to take to install if your system is not ready. For example, if you do not have your instance backed up, the k3s_preinstall.sh script will inform you that the backups are missing and tell you to perform a backup before proceeding. Once the script has completed, it saves the output to the NEXT_STEPS.txt file.
   • If you have made an error during the pre-installation process, perform the pre-installation cleanup commands. and start over.
   • It is safe to run the ./k3s_preinstall.sh command as many times as is necessary. No destructive actions are performed on your cluster by running the pre-install script.
9. Verify that all of the files produced by running the pre-installation scripts were successfully generated. If you cannot locate all of the files, stop here and consult your Workbench support team to diagnose the failures and complete pre-installation successfully.

​

Installation

The following steps detail the conventional output of what the k3s_preinstall.sh script will tell you to do. However, the script makes adjustments to these steps based on your local environment and outputs them to the NEXT_STEPS.txt file. Review the file and, if the output diverges from these instructions, follow the instructions in the file instead.

1. Log out of the server node, then log back in to pick up the environment changes that were made during the pre-installation process.
2. Pause the application by running the following command:
   ae_pause.sh
   
3. Uninstall Gravity and Workbench by running the following commands on each node, starting with the worker nodes and ending with the control plane node:
   sudo gravity system uninstall
   sudo reboot
   
   This removes Gravity and the Workbench application from the cluster, but leaves the Workbench data in place!
   Reboot each time you uninstall a node!
4. Navigate to the anaconda-enterprise-k3s-* directory and install K3s by running the following command:
   ./k3s_install.sh
   
   This script installs and starts K3s, then uploads the Workbench images. The image upload process can take up to 30 minutes to complete, depending on your setup.
5. Once the image uploads begin, you can continue with installation by opening a second terminal and navigating back to the anaconda-enterprise-k3s-* directory, or you can wait until the images are fully uploaded to continue.
6. If your cluster has multiple nodes, add them now by running the following command for each node:
   # Replace <PRIVATE_IP> with the nodes private network IP address
   ./k3s_addnode.sh <PRIVATE_IP>
   
7. Install the SSL certificates saved by the extract_config.sh script by running the following command:
   kubectl create -f anaconda-enterprise-certs.json
   
8. Verify that the k3s_values.yaml contains the expected Workbench configuration values.
   Take your time and verify everything is correct!
9. If necessary, restore your instance’s manually created anaconda-persistence volume at this time by running the following command:
   # Replace <FILE> with the anaconda-persistence volume file name
   kubectl create -f <FILE>.yaml
   
   If you have any other volumes, provision them now as well.
10. Once the Workbench images are fully loaded, install the Workbench application in paused mode by running the following command:
    helm install -f k3s_values.yaml --set global.paused=true anaconda-enterprise Anaconda-Enterprise/
    
11. Restore the system configurations from the backup you took earlier by running the following command:
    ae_restore.sh ae5_config_db_<YYYYMMDDHHMM>.tar.gz
    
12. Unpause the backed up deployments and scheduled jobs by running the following command:
    ae_unpause.sh
    
13. Monitor your cluster’s resources while they stabilize to ensure there are no unexpected issues by running the following command:
    watch kubectl get pods
    
    Allow time to pass for the pods to appear and move to the Ready and Running states.
14. If you are upgrading from 5.5.x or older, you must complete the manual Keycloak upgrade process at this time.
15. Open a browser and navigate to your instance of Workbench.
16. Log in to Workbench and explore the platform to verify that all of your projects, deployments, and scheduled jobs are present.