Prerequisites

• Connectware is already installed and configured.
• Connectware license key.
• Helm V3 installed (Helm | Installing Helm).
• kubectl installed (Install Tools).
• kubectl configured with the current context pointing to your target cluster (Configure Access to Multiple Clusters).
• A Kubernetes namespace that will be used.

Adding the Helm chart repository

Add the Cybus connectware-helm repository to your local Helm installation to use the connectware-agent Helm chart to install Connectware agents in Kubernetes:

helm repo add cybus https://repository.cybus.io/repository/connectware-helm

To verify that the Helm chart is available you can execute a Helm search:

helm search repo connectware-agent

NAME	CHART VERSION	APP VERSION	DESCRIPTION
cybus/connectware-agent standalone agents	1.0.0	1.1.5	Cybus Connectware

Creating your values.yaml file

As with all Helm charts, the connectware-agent chart is configured using a YAML file. This file can have any name, however we will refer to it as the values.yaml file.

Create this file to start configuring your agent installation by using your preferred editor:

vi values.yaml

Installing a single agent

To quickly install a single agent you only need to add your Connectware license key to your values.yaml file as the Helm value licenseKey:

licenseKey: <your-connectware-license-key>

You can now use this file to deploy your Connectware agent in a Kubernetes namespace of your choice:

helm upgrade -i connectware-agent cybus/connectware-agent -f values.yaml -n <namespace>

Release „connectware-agent“ does not exist. Installing it now.

NAME: connectware-agent

LAST DEPLOYED: Mon Mar 13 14:31:39 2023

NAMESPACE: connectware

STATUS: deployed

REVISION: 1

TEST SUITE: None

NOTES:

Thank you for using the Cybus Connectware agent Helm chart!

For additional information visit https://cybus.io/

Number of agents: 1

——————–

– agent

If any of these agents are new, please remember to visit Connectware’s client registry to set up the connection to Connectware.

Hint: If you have agents stuck in a status other than „Running“, you need to delete the stuck pods before a pod with your new configuration will be created.

This will start a single Connectware agent named “agent”, which will connect to a Connectware installation deployed in the same namespace. Unlock the Client Registry in your Connectware admin UI to connect this agent. Refer to Client Registry — Connectware 1.3.2 documentation to learn how to use the Client Registry to connect agents.

You can repeat the same command to apply any changes to your values.yaml file configuration in the future.

Changing the target Connectware

If you are not deploying the agent in the same Kubernetes namespace, or even inside the same Kubernetes cluster, you need to specify the hostname under which Connectware is reachable for this agent.

In the default configuration, the following network ports on Connectware must be reachable for the agent:

• TCP/443
• TCP/1883

Specify the hostname of Connectware to which the agent connects to by setting the Helm value connectwareHost inside the protocolMapperAgentDefaults context of your values.yaml file. For Connectware deployments in a different Kubernetes namespace this is “connectware.<namespace>”.

Example:

licenseKey: <your-connectware-license-key> protocolMapperAgentDefaults: connectwareHost: connectware.cybus # adjust to actual hostname of Connectware

Connectware with separate control-plane-broker

To connect to a Connectware that uses the separate control-plane-broker, you need to set the Helm value controlPlaneBrokerEnabled to true inside the protocolMapperAgentDefaults section of your values.yaml file.

Example:

licenseKey: <your-connectware-license-key> protocolMapperAgentDefaults: connectwareHost: connectware.cybus # adjust to actual hostname of Connectware controlPlaneBrokerEnabled: true

Note: This adds TCP/1884 to required network ports.

Installing multiple agents with a single Helm installation

You can use the agent chart to install multiple Connectware agents. Every agent you configure needs to be named using the Helm value name in a collection entry inside the context protocolMapperAgents. This way, the default name “agent” will be replaced by the name you give the agent.

Example:

licenseKey: <your-connectware-license-key> protocolMapperAgentDefaults: connectwareHost: connectware.cybus # adjust to actual hostname of Connectware protocolMapperAgents: - name: bender-robots - name: welder-robots

This will deploy two Connectware agents, named “bender-robots” and “welder-robots”, both of which will contact the Client Registry of Connectware inside the Kubernetes namespace “cybus”, as described in Client Registry — Connectware 1.3.2 documentation

by Klaus Pittig
updated by Philipp Geschke

Introduction

This quick start guide describes the steps to install the Cybus Connectware onto a Kubernetes cluster.
Please consult the article Installing Cybus Connectware for the basic requirements to run the software, like having access to the Cybus Portal to acquire a license key.

The following topics are covered by this article:

• Prerequisites
• Quick Installation Guide
• The Cybus Connectware Helm Chart
  • System requirements, resource limits and permissions
  • Specialties 
  • Deploying Protocol Mapper Agents

Prerequisites

Please consult the prerequisites chapter in the article Installing Cybus Connectware, we assume that you are already familiar with the Cybus Portal and obtaining a license key or a license file.

This guide does not introduce Kubernetes, Docker, containerization or tooling knowledge, we expect the system admin to know about their respective Kubernetes environment, which brings – besides wellknown standards – a certain specific custom complexity, e.g. the choice of certain load balancers, the management environment, storage classes and the like, which are up to the customer’s operations team and should not affect the reliability of Cybus Connectware deployed there, if the requirements are met.

Besides a Kubernetes cluster the following tools and resources are required:

• the kubectl CLI 
• the Helm V3 CLI
• Ability to reach your Kubernetes cluster
• Ability to download Docker images from registry.cybus.io directly or through a mirror/proxy to your Kubernetes cluster nodes, if the internet connection is restricted.

Quick Installation Guide

To be able to start with Cybus Connectware on a Kubernetes cluster use the prepared helm chart please follow the following steps:

1. Add the public Helm V3 repository to the local configuration using the Helm CLI:
   helm repo add cybus https://repository.cybus.io/repository/connectware-helm
2. Update repository and verify available Connectware Helm Chart releases (e.g. 1.1.0)
   helm repo update
helm search repo connectware [-l]
3. Create a file called values.yaml. This file will be used to configure your installation of Connectware. Initially fill this file with this YAML content:

global:
  licensekey: <YOUR-CONNECTWARE-LICENSE-KEY>
  setImmutableLabels: true
  broker:
    clusterSecret: <SOME-RANDOM-SECRET-STRING>

• If your Kubernetes cluster does not specify a default StorageClass that provides ReadWriteOnce and ReadWriteMany access modes, please also set the value global.storage.storageClassName to a StorageClass that does

storage:
    storageClassName: “san-storage” # example value

• Have a look at the default values by extracting them into a file called default-values.yaml and checking if you want to make further adjustments. It is, for example, possible that you need to adjust the resource request/limit values for smaller test clusters. In this case copy and adjust the global.podResources section from default-values.yaml to values.yaml.
  helm show values cybus/connectware > default-values.yaml
• Optionally perform a dry-run with debug options to validate all settings:
  helm install <YOURDEPLOYMENTNAME> cybus/connectware -f ./values.yaml --dry-run --debug -n<YOURNAMESPACE> --create-namespace
  (shows user supplied values, along with computed values and the rendered template)
  Example: helm install connectware cybus/connectware -f ./values.yaml --dry-run --debug -ncybus --create-namespace
• Start the installation through Helm:
  helm install <YOURDEPLOYMENTNAME> cybus/connectware -f ./values.yaml --n<YOURNAMESPACE> --create-namespace
• Subsequent updates can be executed by
  helm upgrade <YOURDEPLOYMENTNAME> cybus/connectware -f ./values.yml -n<YOURNAMESPACE>

Important values.yml parameters

When taking a look at the default-values.yaml file you should check out these important values within the global section:

• The licensekey value handles the licensekey of the connectware installation. This needs to be a production license key. This parameter is mandatory unless you set licenseFile
• The licenseFile value is used to activate the Connectware in offline mode. The content of a license file downloaded from the Cybus Portal has to be set (this is a single line of a base64 encoded json object)
• The current version of the Helm chart always installs the most recent version of Connectware. You can override the connectware image source and version using the image section. 
• The broker section specifies MQTT broker related settings:
  • broker.clusterSecret: the authentication secret for the MQTT broker cluster
  • broker.replicaCount: the number of broker instances
• The loadBalancer section allows pre-configuration for a specific load balancer
• The podResources set of values allows you to configure the number of CPU and memory resources per pod; by default some starting point values are set, but depending on the particular use case they need to be tuned in relation to the expected load in the system, or reduced for test setups
• The protocolMapperAgents section allows you to configure additional protocol-mapper instances in Agent mode. See the documentation below for more details

Important notes on setting values when upgrading to newer versions of the chart

Helm allows setting values by both specifying a values file (using -f or --values) and the --set flag. When upgrading this chart to newer versions you should use the same arguments for the Helm upgrade command to avoid conflicting values being set for the Chart; this is especially important for the value of global.broker.clusterSecret, which would cause the nodes not to form the cluster correctly, if not set to the same value used during install or upgrade.

For more information about value merging see the respective Helm documentation.

Using the Connectware

After following all the steps above Cybus Connectware is now installed. You can access the Admin UI by opening your browser and entering the Kubernetes application URL https://<external-ip> with the initial login credentials username admin and password admin.

To determine this data, the following kubectl command can be used:
kubectl get svc connectware --namespace=<YOURNAMESPACE> -o jsonpath={.status.loadBalancer.ingress}

Should this value be empty your Kubernetes cluster load-balancer might need further configuration, which is beyond the scope of this document, but you can take a first look at Connectware by port-forwarding to your local machine:
kubectl --namespace=<YOURNAMESPACE> port-forward svc/connectware 10443:443 1883:1883 8883:8883

You can now access the admin UI at https://localhost:10443/

If you would like to learn more how to use the Connectware, check out our docs at https://docs.cybus.io/ or see more guides here.

The Cybus Connectware Helm Chart

The Kubernetes version of Cybus Connectware comes with a Helm Umbrella chart, describing the instrumentation of the Connectware images for deployment in a Kubernetes cluster.

It is publicly available in the Cybus Repository for download or direct use with Helm.

System requirements

Cybus Connectware expects a regular Kubernetes cluster and was tested for Kubernetes 1.22 or higher.

This cluster needs to be able to provide load-balancer ingress functionality and persistent volumes in ReadWriteOnce and ReadWriteMany access modes provided by a default StorageClass unless you specify another StorageClass using the global.storage.storageClassName Helm value.

For Kubernetes 1.25 and above Connectware needs a privileged namespace or a namespace with PodSecurityAdmission labels for warn mode. In case of specific boundary conditions and requirements in customer clusters, a system specification should be shared to evaluate them for secure and stable Cybus Connectware operations.

Resource limits

Connectware specifies default limits for CPU and memory in its Helm values that need to be at least fulfilled by the Kubernetes cluster for production use. Variations need to be discussed with Cybus, depending on the specific demands and requirements in the customer environment, e.g., the size of the broker cluster for the expected workload with respect to the available CPU cores and memory.

Smaller resource values are often enough for test or POC environments and can be adjusted using the global.podResources section of the Helm values.

Permissions

To be able to run Cybus Connectware in Kubernetes clusters  two new RBAC roles will be deployed through the Helm chart and will provide Connectware with these namespaced permissions:

Broker:

resource(/subresource)/action	permission
pods/list	list all containers get status of all containers
pods/get pods/watch	inspect containers
statefulsets/list	list all StatefulSets get status of all StatefulSets
statefulsets/get statefulsets/watch	inspect StatefulSets

Container/Pod-manager:

resource(/subresource)/action	permission
pods/list	list all containers get status of all containers
pods/get pods/watch	inspect containers
pods/log/get pods/log/watch	inspect containers get a stream of container logs
deployments/create	create Deployments
deployments/delete	delete Deployments
deployments/update deployments/patch	to restart containers (since we rescale deployments)

Specialties

The system administrator needs to be aware of certain characteristics of the Connectware deployment:

• The Helm Chart requires a licensekey to download images from the Cybus registry, which requires a proxy or direct connection to the registry.cybus.io host.
• To download a license file for the key a connection to portal.cybus.io is required.
  • This can be skipped for an offline setup by defining the licenseFile parameter in the Helm chart values (see licenseFile  above)
• Dynamic scaling is just possible for the broker and control-plane-broker StatefulSet.
• Multiple ProtocolMapper instances are possible by explicit definition of additional protocolMapperAgents in the Helm chart values (see below)
• In case of a MetalLB the values.yaml can define an addressPoolName to assign an external IP address range managed by MetalLB through the Helm value
  global.loadBalancer.addressPoolName or by setting the
  metallb.universe.tf/address-pool annotation using the
  global.ingress.service.annotations Helm value

Deploying Protocol-Mapper Agents

The default-values.yaml file contains a protocolMapperAgents section representing a list of Connectware agents to deploy. The general configuration for these agents is the same as described in the Connectware documentation.

You can copy this section to your local values.yaml file to easily add agents to your Connectware installation

The only required property of the list items is name; if only this property is specified the chart assumes some defaults:

• CYBUS_AGENT_NAME is set to the same value as that of name
• CYBUS_MQTT_HOST is set to connectware which is the DNS name of the Connectware Note: this DNS name must be a DNS name reachable by each of the agents being deployed. In case of some deployment distribution strategy of the target Kubernetes cluster this name has to be configured accordingly.
• storageSize is set to 40 MB by default. The agents use some local storage which needs to be configured based on each use case. If a larger number of services is going to be deployed, this value should be specified and set to bigger values.

You can check out the comments of that section in default-values.yaml to see further configuration options.

You can find further information in the general Connectware Agent documentation.