Kubernetes | 30 November 2023
Upgrading Connectware on Kubernetes
Important: Connectware currently does not support hit-less upgrades. You may experience a service degradation during upgrading. Make sure to take an appropriate maintenance window into account when upgrading.
Prerequisites for upgrading Connectware
Before upgrading Connectware, make sure that you meet the following prerequisites:
- The Kubernetes command line tool kubectl is configured and has access to the target installation.
- Helm version 3 is installed on your system.
- The following information is available:
Connectware version number that you want to upgrade to
Make sure that you know the exact version number of the Connectware version that you want to upgrade to.
For the code examples in this documentation, we use the variable <target-version> to refer to the Connectware version that you want to upgrade to.
Pulling updated Helm information
You must update the Helm repository cache to make sure that you receive the latest Connectware version.
- To update your repository cache for the Connectware Helm chart, enter the following command:
helm repo update
Reviewing the Connectware changelog
Before you upgrade to a new Connectware version, we recommend that you read the changelog to find out about new features, bug fixes, and changes of the Connectware version that you want to upgrade to.
- To open the Connectware changelog, click here: https://docs.cybus.io/latest/user/changelog.html
Reviewing the readme file
Before you upgrade to a new Connectware version, read the readme file of the Connectware version that you want to upgrade to for additional upgrade instructions.
- To open the readme file, enter the following command:
<code><code>helm show readme <repo-name>/connectware --version <target-version></code></code>
Code-Sprache: YAML (yaml)Comparing Helm configurations between Connectware versions
With a new Connectware version, there might be changes to the default Helm configuration values. We recommend that you compare the default Helms values of your current Connectware version with the default Helm values of your target Connectware version.
- To display the new default values, enter the following command:
<code>helm show values <repo-name>/connectware --version <target-version></code>
Code-Sprache: YAML (yaml)- To display which Connectware default values have changed between your current version and your target version, enter the following command:
<code>diff <(helm show values <repo-name>/connectware --version <current-version>) <(helm show values <repo-name>/connectware --version <target-version></code>
Code-Sprache: YAML (yaml)Example
diff <(helm show values cybus/connectware --version 1.1.0) <(helm show values cybus/connectware --version 1.1.1)
83c83
< version: 1.1.0
---
> version: 1.1.1
Code-Sprache: YAML (yaml)In this example, only the image version has changed. However, if any of the Helm value changes are relevant to your setup, make the appropriate changes.
- To override default Helm values, add the custom Helm value to your local values.yaml file.
Adjusting Helm values
When you have reviewed the necessary information, adjust your configuration in your <values.yaml> file. Not every upgrade requires adjustments.
If you specified which image version of Connectware to use by setting the Helm value global.image.version you will need to update this to <target-version>. Unless you have a specific reason to use a specific image version, we recommend not setting the Helm value.
Verifying your backups
Make sure that you store backups of your setup. This allows you to restore a previous state if necessary.
Your backups must consist of the following files:
- All Kubernetes PersistentVolumes that Connectware uses
- Your Connectware database
- To create a backup of the Connectware database via the Connectware Admin UI, select System > Backup and Restore and click CREATE BACKUP. For more information on backing up and restoring, see the Connectware documentation.
- Your values.yaml file
- All service commissioning files
Depending on your local infrastructure, it may be necessary to back up additional files.
Starting the Connectware upgrade
Once you have all the information that you need to upgrade your Connectware, you can start the upgrade process. The following sections will guide you through monitoring the upgrade, as well as what to do on failed upgrades.
- To upgrade Connectware, enter the following command:
<code>helm upgrade -n <namespace> <installation-name> <repo-name>/connectware --version <target-version> -f <values.yaml></code>
Code-Sprache: YAML (yaml)Optionally you can use the --atomic --timeout 10m command line switch, which will cause Helm to wait for the result of your upgrade and perform a rollback when it fails. We recommend setting the timeout value to at least 10 minutes, but because the time it takes to complete an upgrade strongly depends on your infrastructure and configuration you might have to increase it further.
Result: The newly generated workload definitions are applied to your Kubernetes cluster and your Connectware pods are replaced.
Verifying the Connectware upgrade
You can monitor the Connectware upgrade progress to verify that everything runs smoothly, to know when the installation is successful, or to investigate potential issues.
Monitoring the Connectware upgrade
The Connectware upgrade can take a few minutes. To monitor the upgrade process, do one of the following:
- To monitor the current status of the upgrade process, enter the following command:
<code>kubectl get pods -n <namespace></code>
Code-Sprache: YAML (yaml)- To monitor the continuous progress of the upgrade process, enter the following command:
<code>while [ True ]; do clear; kubectl get pod -n <namespace>; sleep 5; done</code>
Code-Sprache: YAML (yaml)- To stop monitoring the continuous progress of the upgrade process , press Ctrl+C.
Pod stages during the Connectware upgrade
During the Connectware upgrade, the pods go through the following stages:
- Terminating
- Pending
- PodInitializing
- ContainerCreating
- Init:x/x
- Running
When pods reach the STATUS Running, they go through their individual startup before reporting as Ready. To be fully functional, all pods must reach the STATUS Running and report all their containers as ready. This is indicated by them showing the same number on both sides of the / in the column READY.
Example
<code>$ kubectl get pod -n <namespace></code>
Code-Sprache: YAML (yaml)| NAME | READY | STATUS | RESTARTS | AGE |
|---|---|---|---|---|
| admin-web-app-7cd8ccfbc5-bvnzx | 1/1 | Running | 0 | 3h44m |
| auth-server-5b8c899958-f9nl4 | 1/1 | Running | 0 | 3m3s |
| broker-0 | 1/1 | Running | 0 | 3h44m |
| broker-1 | 1/1 | Running | 0 | 2m1s |
| connectware-7784b5f4c5-g8krn | 1/1 | Running | 0 | 21s |
| container-manager-558d9c4cbf-m82bz | 1/1 | Running | 0 | 3h44m |
| doc-server-55c77d4d4c-nwq5f | 1/1 | Running | 0 | 3h44m |
| ingress-controller-6bcf66495c-l5dpk | 1/1 | Running | 0 | 18s |
| postgresql-0 | 1/1 | Running | 0 | 3h44m |
| protocol-mapper-67cfc6c848-qqtx9 | 1/1 | Running | 0 | 3h44m |
| service-manager-f68ccb767-cftps | 1/1 | Running | 0 | 3h44m |
| system-control-server-58f47c69bf-plzt5 | 1/1 | Running | 0 | 3h44m |
| workbench-5c69654659-qwhgc | 1/1 | Running | 0 | 15s |
At this point Connectware is upgraded and started. You can now make additional configurations or verify the upgrade status in the Connectware Admin UI.
For more information on the Connectware Admin UI, see the Connectware documentation.
Troubleshooting pod stages
If a pod is in another state than expected or if it is stuck at a certain stage for more than three minutes, there might be an issue.
- To investigate the pod status, enter the following command:
<code>kubectl describe pod <pod-name></code>
Code-Sprache: YAML (yaml)For help on solving issues, see Troubleshooting Connectware on Kubernetes.
Rolling back the Helm upgrade
If the Helm upgrade fails, and it is not possible to immediately identify and fix the problem, you can roll back Helm upgrades using the helm rollback command.
To perform the rollback you need to know the current REVISION of your installation. Use the command helm list -n <namespace>, and note down the value for REVISION displayed in the row of your Connectware installation. In the next section you will use this value and decrement one to restore the previous REVISION. For example, if the REVISION displayed is 8, you will use 7 in the helm rollback command:
<code>helm rollback --wait -n <namespace> <installation-name> <REVISION - 1></code>
Code-Sprache: YAML (yaml)Note: This will roll back your Helm upgrade and start your previous version of Connectware. It is possible that you will need to perform a restore of Connectware, if there were modifications made by the attempted upgrade