How to | 13 August 2020
Service Basics
Prerequisites
It’s a good idea to make yourself familiar with the following topics before commencing this lesson:
Introduction
This lesson will cover the basic concepts of Cybus Services and how they relate to the Connectware. We will focus on how applications deployed as services can take advantage of the data broker for accessing output from data sources on the Connectware, the management cluster for gaining access to that output and the single entry point for providing new interfaces where users can take advantage of processed data.
What is a Service
While the Connectware gives interfaces for individually performing all actions (like managing users, setting permissions, starting containerized applications, etc.) the idea of services is to bundle all this activities into a single point of configuration and execution. This means that when a service is enabled it will perform a set of operations on the Connectware and when disabled these operations will be removed. Below you will find some examples of possible service use cases.
Preprocessing
Think of a simple machine that produces a metal part. Whether a metal part is being made at any one point of time may not be the important but the average amount being made over a given time period may be useful for knowing if the machine is underperforming. We can accomplish this by connecting the machine to the Connectware, and then deploying a service along side that takes the raw data from the data broker, calculates the average over a given period of time, then uploads the result to the data broker for some other application to consume. This type of application is classified as a preprocessor as it allows us to perform some operation on the data before it is consumed elsewhere.
Device controller
Think of a machine that drills holes in a piece of material. When the machine is drilling we want to set a lamp to be green, when the machine is between drilling we want to set a lamp to be yellow and when the machine is powered off we want the light to be red. We can connect both the machine and the lamp to the Connectware, then read the status of the machine from the data broker into a service. This service can perform a check on the status and then write out to the data broker on a topic that controls the light to change colors. This type of application is classified as a Device controller as we are using input data to write to a device connected to the Connectware.
Data visualization
Think of an assembly line that outputs new products ready to be shipped. We can connect the sensors of the assembly to the Connectware and build a service that provides a web dashboard which outputs different graphs for the average output, speed, temperature and power used throughout the day. Administrators can then sign in to the Connectware and view this dashboard to see the status of the assembly line. This type of application is classified as a Data Visualization as it takes data from the Connectware and provides visualizations to allow easier consumption of important data.
Much more
These are only a few examples of the types of services that Cybus Connectware can deploy. As you will see below the Connectware is built on top of Docker, so as long as your application can be containerized it can be deployed on the Connectware.
How to create a Basic Service
Services are installed using a commissioning file. Within this text-based YAML file, all so-called resources like connections, endpoints, users, permissions and containerized applications a service uses are listed. Below you will find an example service commissioning file that deploys a Grafana instance which we will configure to show data provided by an OPC UA Server. The service will use one Cybus specific container to map MQTT to InfluxDB but the rest of the containers were taken from other projects and are easily integrated. If any section of the service commissioning file needs clarification please feel free to visit the Connectware Docs.
Description & Metadata
The first two sections of the service commissioning file give more general information. In the section description you can give a short description of the service which will also be displayed in the service’s details view on the Connectware Admin User Interface (Admin UI). The section metadata contains the meta information for the service that is being installed. Here we set a name of the service, version number, provide an icon for the thumbnail of the service, specify a provider and a related homepage.
#------------------------------------------------------------------------------
# CYBUS SERVICE COMMISSIONING
#------------------------------------------------------------------------------
description: |
Service Commissioning File Example
Cybus Learn - Service Basics
https://learn.cybus.io/lessons/service-basics/
metadata:
name: Service Basics Example
version: 0.0.1
icon: https://www.cybus.io/wp-content/uploads/2019/03/Cybus-logo-Claim-lang.svg
provider: Cybus GmbH
homepage: https://www.cybus.ioParameters
To make a service configurable, we can specify parameters. Parameters are like referable variables whose values are defined by the user every time a service is to be installed or reconfigured. When applying a commissioning file in the Connectware, the user is asked to enter custom values for the parameters or to confirm the default values.
In this example we use parameters to make the OPC UA server address configurable for the case it would move to another location.
#------------------------------------------------------------------------------
# Parameters
#------------------------------------------------------------------------------
parameters:
opcuaHost:
type: string
description: OPC UA Host Address
default: opcuaserver.com
opcuaPort:
type: integer
description: OPC UA Host Port
default: 48010Resources
In the Resources section we declare every resource that is needed for our service. All resources like connections, endpoints, users, permissions and containerized applications are configured here. The first resources we define are a user and and role granting permissions to our service.
#------------------------------------------------------------------------------
# Resources
#------------------------------------------------------------------------------
resources:Users & Roles
To keep your infrastructure safe, some parts of the service have to use credentials for authorization on MQTT or HTTP interfaces. Within the commissioning file you can define users and roles which will be created on installation and can then be used by the service to access specific data or APIs.
A part of our service, which will access data via MQTT and forward it to InfluxDB, will have to authorize on the data interface. So we create the user dataMonitor with the global parameter Cybus::MqttPassword as a password, which holds an auto-generated, valid password for the Connectware’s internal MQTT broker, so we do not have to hardcode or manually configure it on installation. We assign the role dataReaderRole to it which is created afterwards and equipped with read permissions for any MQTT topic under building-automation (specified with the wildcard #).
To learn more about the details of creating users and roles check out the lesson on User Management.
#----------------------------------------------------------------------------
# USERS
#----------------------------------------------------------------------------
# purpose: Forwards data to InfluxDB and provides a Grafana instance
dataMonitor:
type: Cybus::User
properties:
password: !ref Cybus::MqttPassword
roles:
- !ref dataReaderRole
permissions: []
#----------------------------------------------------------------------------
# ROLES
#----------------------------------------------------------------------------
# purpose: Reads data from specified MQTT topic
dataReaderRole:
type: Cybus::Role
properties:
permissions:
- resource: 'building-automation/#'
operation: read
context: mqttConnections & Endpoints
In this lesson we will collect some simulated data from a public OPC UA server. We define a connection and an endpoint resource for this. For the connection we just need to specify the protocol as well as the server’s host address and port, which we define by referencing the previously declared parameters. For the endpoints we define three resources, each relating to a data source on the OPC UA server identified through the nodeId.
To learn more about the details of defining connections and endpoints utilizing various protocols, check out the other lessons on Cybus Learn. To learn more about the setup of an OPC UA connection and endpoints, check out the article How to connect and integrate an OPC UA server.
#----------------------------------------------------------------------------
# CONNECTIONS
#----------------------------------------------------------------------------
opcuaConnection:
type: Cybus::Connection
properties:
protocol: Opcua
connection:
host: !ref opcuaHost
port: !ref opcuaPort
#username: myUsername
#password: myPassword
#----------------------------------------------------------------------------
# ENDPOINTS
#----------------------------------------------------------------------------
Humidity:
type: Cybus::Endpoint
properties:
protocol: Opcua
connection: !ref opcuaConnection
subscribe:
nodeId: ns=3;s=AirConditioner_1.Humidity
PowerConsumption:
type: Cybus::Endpoint
properties:
protocol: Opcua
connection: !ref opcuaConnection
subscribe:
nodeId: ns=3;s=AirConditioner_1.PowerConsumption
Temperature:
type: Cybus::Endpoint
properties:
protocol: Opcua
connection: !ref opcuaConnection
subscribe:
nodeId: ns=3;s=AirConditioner_1.TemperatureMappings
The data collected by our endpoints should be available to our service on the MQTT data interface provided by the internal Connectware MQTT Broker. The data is already available on service related MQTT topics (under services) but we need the data in a place which our newly created user can access (under building-automation). So we create a mapping that subscribes to our endpoints and publishes any received data to the desired MQTT topics.
For more information on the creation of mappings again check out the lesson How to connect and integrate an OPC UA server.
#----------------------------------------------------------------------------
# MAPPINGS
#----------------------------------------------------------------------------
mapping:
type: Cybus::Mapping
properties:
mappings:
- subscribe:
endpoint: !ref Humidity
publish:
topic: 'building-automation/airconditioner/1/humidity'
- subscribe:
endpoint: !ref PowerConsumption
publish:
topic: 'building-automation/airconditioner/1/power-consumption'
- subscribe:
endpoint: !ref Temperature
publish:
topic: 'building-automation/airconditioner/1/temperature'Volumes
A volume is a resource that represents a storage space and can be associated to containers. We want to utilize two containers, which will need additional storage space, so we create a volume for each of them.
#----------------------------------------------------------------------------
# VOLUMES
#----------------------------------------------------------------------------
grafanaVolume:
type: Cybus::Volume
influxdbVolume:
type: Cybus::VolumeFrontends
In the frontends section we define the interfaces that the service will provide on a Connectware instance. Concerning the frontend we want to declare two resources. First we create an ingress route which will then point to a specific port in the service container. This allows services to provide web dashboards and REST APIs which can then be accessed through the HTTPS interface of the Cybus Connectware. In this case we define an HTTP interface and point it to port 3000 on the Grafana container which will allow us to access the dashboards. Second we define a link to the ingress route, which will simply provide a button named Dashboard on our service details view in the Connectware Admin UI. To learn more details about ingress route resources check out the Connectware Docs.
#----------------------------------------------------------------------------
# FRONTENDS
#----------------------------------------------------------------------------
# Grafana
grafanaURL:
type: Cybus::IngressRoute
properties:
container: !ref genericGrafana
type: http
slug: grafana
target:
path: '/'
port: 3000
dashboard:
type: Cybus::Link
properties:
name: Dashboard
ingressRoute: !ref grafanaURL
href: ''Containers
The containers section comprises the Docker Containers the service will run. These containers can either come from the Official Docker Registry or from the Cybus Registry. That means any application that is deployed on the Connectware can take full advantage of all the containerized software on Docker Hub and your custom containerized software delivered securely through the Cybus Registry. In the example below we pull the official Grafana and InfluxDB containers from Docker Hub and then use a custom Cybus container to map the MQTT data to InfluxDB. The different options that can be used when configuring these containers can be found in the Connectware Docs and concerning the container-specific environmental variables defined under the property environment you should check out the container’s documentation .
#----------------------------------------------------------------------------
# CONTAINERS
#----------------------------------------------------------------------------
influxdb:
type: Cybus::Container
properties:
image: registry.cybus.io/cybus-services/influxdb:1.7.8-alpine
volumes:
- !sub '${influxdbVolume}:/var/lib/influxdb'
environment:
INFLUXDB_DB: generic
genericGrafana:
type: Cybus::Container
properties:
image: registry.cybus.io/cybus-services/generic-grafana:1.3.0
volumes:
- !sub '${grafanaVolume}:/var/lib/grafana'
environment:
GF_SERVER_ROOT_URL: !sub '/services/${Cybus::ServiceId}/grafana'
GF_AUTH_ANONYMOUS_ENABLED: true
INFLUX_HOST: !ref influxdb
INFLUX_PORT: 8086
INFLUX_DB: generic
influxdbPush:
type: Cybus::Container
properties:
image: registry.cybus.io/cybus-services/cybus-influxdb-push:2.0.1
environment:
MQTT_HOST: !ref Cybus::MqttHost
MQTT_USERNAME: !sub '${Cybus::ServiceId}.dataMonitor'
MQTT_PASSWORD: !ref Cybus::MqttPassword
MQTT_SUBSCRIPTIONS: 'building-automation/#'
INFLUXDB_HOST: !ref influxdbHow to install a Service
Now that we know what a service is and we have configured our own example we can install it on the Connectware. The service commissioning file can be found on GitHub.
Install Service
1) Open the Admin UI of your Connectware instance and navigate to the services section.
2) Click the plus-button in the upper right corner to add a service.
3) Select the service-example-commissioning-file.yml on your computer and click Install to confirm the default values and start the installation.
4) Click the now existing entry of your service in the list to open the details view.
5) Click the enable button on your Service.
Explore Service
Now that you have successfully installed and enabled a service we can use the provided dashboard and configure Grafana to visualize our simulation data.
1) First click the Dashboard button on the service detail view.
2) The welcome screen of Grafana has now opened in a new tab. Initially you need to log in to modify any dashboards. Click the Sign In button in the bottom left corner and log in with your Admin UI credentials.
3) As soon as you are logged in and back to the welcome screen go to the side menu and create a new dashboard.
4) Choose Add Query from the new panel.
5) Through service commissioning our InfluxDB is already connected to Grafana and selected as default data base. Click on select measurement and a list will present the data to you that we mapped on the MQTT topics.
6) With the four circles on the left side you can switch between different setting categories and for example edit the name of the panel.
7) In the upper right corner you can choose the time range and refresh rate. Next to this click the Save dashboard button, choose a name for the dashboard and click Save.
8) Clicking the highlighted Add panel button you can extend your dashboard with more panels like that.
The query editor for Grafana is incredibly powerful and can do much more then we will show in this tutorial. For more details read the documentation here.
Summary
In this lesson we learned what services are and made a basic Grafana service whose configuration gave a short demonstration of the various resources a service can utilize and how they are configured to create a consistent context of cooperation. We then configured Grafana to visualize humidity data from our simulation OPC UA server.
This article should have given you an overview over the possibilities that services offer in terms of interconnectivity. Knowing the methods of utilizing containers, including data sources, organizing their data flow and managing their access permissions should give you a hint of how you can structure the first own commissioning file for your Connectware!
Going further
To learn more about the Cybus Connectware check out our Connectware Docs or read more lessons here on Cybus Learn. To learn more on using the Grafana service read the Grafana Documentation.
Check out our media to this article:
back to Cybus Learn