How to | 03 Januar 2022
Installing Cybus Connectware on Kubernetes clusters
Introduction
This quick start guide describes the steps to install the Cybus Connectware on Kubernetes clusters.
Please consult the article Installing Cybus Connectware for the basic requirements to run the software.
The following topics are covered by this article:
- Prerequisites
- Quick Installation Guide
- The Cybus Connectware Helm Chart
- System requirements, resource limits and permissions
- Specialties
- Deploying Protocol Mapper Agents
Prerequisites
Please consult the prerequisites chapter in the article Installing Cybus Connectware, we assume that you are already familiar with the Cybus Portal and obtaining a license key and a license file.
This guide does not introduce Kubernetes, Docker, containerization or tooling knowledge, we expect the system admin to know about their respective Kubernetes environment, which brings – besides wellknown standards – a certain specific custom complexity, e.g. the choice of certain load balancers, the management environment, storage classes and the like, which are up to the customer’s operations team and should not affect the reliability of Cybus Connectware deployed there, if the requirements are met.
Besides a Kubernetes cluster the following tools and resources are required:
- the kubectl CLI
- the Helm V3 CLI
- Ability to reach your Kubernetes cluster
- Ability to download Docker images from registry.cybus.io directly or through a mirror/proxy, if the internet connection is restricted.
Quick Installation Guide
Start with Connectware on Kubernetes Clusters
To be able to start with Cybus Connectware on a Kubernetes cluster use the prepared helm chart please follow the following steps:
- Add the public Helm V3 repository to the local configuration using the Helm CLI:
helm repo add cybus https://repository.cybus.io/repository/connectware-helm
- Update repository and verify available Connectware Helm Chart releases (e.g. 1.7.0)
helm repo update
helm search repo connectware [-l] - Extract the
values.yaml
file from the helm package and define custom values (see specification below). A connectwarelicenseKey
and abroker.clusterSecret
are essential. Beside this at least the cpu/memory resource definitions at the end of the chart values list need to be adjusted for the runtime environment (in case of restrictions).helm show values cybus/connectware > values.yaml
- Optionally perform a dry-run with debug options to validate all settings:
helm install <YOURDEPLOYMENTNAME> <REPONAME>/connectware -f ./values.yaml --dry-run --debug -n<YOURNAMESPACE> --create-namespace
(shows user supplied values, along with computed values and the rendered template)
Example:sudo helm install connectware-1.0 cybus/connectware -f ./values.preview.yaml --dry-run --debug -ncybus --create-namespace
- Then initially execute accordingly:
helm install <YOURDEPLOYMENTNAME> <REPONAME>/connectware -f ./values.yaml --n<YOURNAMESPACE> --create-namespace
- Subsequent updates can be executed by
helm upgrade <YOURDEPLOYMENTNAME> <REPONAME>/connectware -f ./values.yml -n<YOURNAMESPACE>
Important values.yml parameters
The Helm Chart contains a values.schema.json file with more detailed information about these available options:
- The
licenseKey
value handles the licensekey of the Connectware installation. This needs to be a production license key. - The
licenseFile
value is used to activate the Connectware in offline mode. The content of a license file downloaded from the Cybus Portal has to be set (this is a single line of a base64 encoded json object). - The
image
section specify the connectware image source and version - The
broker
section specifies MQTT broker related settings:broker.clusterSecret:
the authentication secret for the MQTT broker clusterbroker.replicaCount:
the number of broker instancesbroker.enableMutualTLS:
true/false for mandatory TLS for client connections
- The
loadBalancer
section allows pre-configuration for a specific load balancer - The
podResources
set of values allows you to configure the number of resources per pod; by default some minimal values are set as a starting point but depending on the particular use case they need to be tuned in relation to the expected load in the system - The
protocolMapperAgents
section allows you to configure additional protocol-mapper instances in Agent mode. See the documentation below for more details
Important notes on setting values when upgrading to newer versions of the chart
Helm allows setting values by both specifying a values file (using -f
or --values
) and the --set
flag. When upgrading this chart to newer versions you should use the same arguments for the Helm upgrade command to avoid conflicting values being set for the Chart; this is especially important for the value of global.broker.clusterSecret
, which would cause the nodes not to form the cluster correctly, if not set to the same value used during install or upgrade.
For more information about value merging see the respective Helm documentation.
Using the Connectware
After following all the steps above Cybus Connectware is now installed. You can access the Admin UI by opening your browser and entering the Kubernetes application URL https://<external-ip>
with the initial login credentials username=admin and password=admin.
To determine this data, the following kubectl command can be used:kubectl get svc connectware --namespace=<YOURNAMESPACE> -o jsonpath={.status.loadBalancer.ingress}
If you would like to learn more how to use the Connectware, check out our docs at https://docs.cybus.io/ or see more guides here.
The Cybus Connectware Helm Chart
The Kubernetes version of Cybus Connectware comes with a Helm Umbrella chart, describing the instrumentation of the Connectware images for deployment in a Kubernetes cluster.
It is publicly available in the Cybus Repository for download resp. direct use with Helm.
System requirements for Kubernetes clusters
Cybus Connectware expects regular Kubernetes clusters and was tested for Kubernetes 1.18 or higher.
In case of specific boundary conditions and requirements in customer Kubernetes clusters, a system specification should be shared to evaluate them for secure and stable Cybus Connectware operations.
Resource limits
Connectware specifies default limits for CPU and memory in the values.yml
, that need to be at least fulfilled by the Kubernetes cluster. Variations need to be discussed with Cybus, depending on the specific demands and requirements in the customer environment, e.g., the size of the broker cluster for the expected workload with respect to the available CPU cores and memory.
The additional file values.preview.yaml
(for limited cluster environments) show some typical resource limit differences, which are defined to guarantee proper operation of Cybus Connectware.
Permissions
To be able to run Cybus Connectware in Kubernetes clusters the implemented compatibility layer requires the following permissions in form of resource[/sub-resource]/action for the required features.
Current required permissions defined in the role definitions for broker and container/pod-manager are:
Broker:
resource(/subresource)/action | permission |
---|---|
pods/list | list all containersget status of all containers |
pods/get pods/watch | inspect containers |
Container/Pod-manager:
resource(/subresource)/action | permission |
---|---|
pods/list | list all containersget status of all containers |
pods/get pods/watch | inspect containers |
pods/log/get pods/log/watch | inspect containersget a stream of container logs |
deployments/create | create containers |
deployments/delete | delete containers |
deployments/update deployments/patch | to restart containers (since we rescale deployments) |
Specialties
The system administrator needs to be aware of certain characteristics of the Connectware deployment:
- It requires a storage class marked as default, otherwise the volumes fail to be deployed.
- In case of Longhorn storage be aware that it is still ReadWriteOnce, but just on node level.
- The Helm Chart requires a licenseKey to download images from the cybus registry, which requires a proxy or direct connection to the registry.cybus.io host.
- To download a license file for the key a connection to portal.cybus.io is required.
- This can be skipped for an offline setup by defining the licenseFile parameter in the helm chart values (see
licenseFile
above) - Dynamic scaling is just possible for the broker deployment.
- Multiple ProtocolMapper instances are possible by explicit definition of additional protocolMapperAgents in the helm chart values.
- In case of a MetalLB the values.yaml can define an addressPoolName to assign an external IP address range managed by MetalLB.
Deploying Protocol-Mapper Agents
The values.yaml
file contains a protocolMapperAgents
section representing a list of Connectware agents to deploy. The general configuration for these agents is the same as described in the Connectware documentation.
The only required property of the list items is name; if only this property is specified the chart assumes some defaults:
- CYBUS_AGENT_NAME is set to the same value as that of
name
- CYBUS_MQTT_HOST is set to
connectware
which is the DNS name of the Connectware Note: this DNS name must be a DNS name reachable by each of the agents being deployed. In case of some deployment distribution strategy of the target Kubernetes cluster this name has to be configured accordingly. storageSize
is set to 40 MB by default. The agents use some local storage which needs to be configured based on each use case. If a larger number of services is going to be deployed, this value should be specified and set to bigger values.
Each entry on the list of agents accepts env
properties as environment variables injected into each agent:
- CYBUS_MQTT_USERNAME
- CYBUS_MQTT_SCHEME
- CYBUS_MQTT_PORT
The complete documentation for these values can be found in the Connectware Agent documentation.