Generic selectors
Exact matches only
Search in title
Search in content
Search in posts
Search in pages
Cybus Learn |   Service Basics

How to | 13 August 2020

Service Basics

By: Jacob Evans

Prerequisites

It’s a good idea to make yourself familiar with the following topics before commencing this lesson:

Introduction

This lesson will cover the basic concepts of Cybus Services and how they relate to the Connectware. We will focus on how applications deployed as services can take advantage of the data broker for accessing output from data sources on the Connectware, the management cluster for gaining access to that output and the single entry point for providing new interfaces where users can take advantage of processed data.

What is a Service

While the Connectware gives interfaces for individually performing all actions (like managing users, setting permissions, starting containerized applications, etc.) the idea of services is to bundle all this activities into a single point of configuration and execution. This means that when a service is enabled it will perform a set of operations on the Connectware and when disabled these operations will be removed. Below you will find some examples of possible service use cases.

Preprocessing

Think of a simple machine that produces a metal part. Whether a metal part is being made at any one point of time may not be the important but the average amount being made over a given time period may be useful for knowing if the machine is underperforming. We can accomplish this by connecting the machine to the Connectware, and then deploying a service along side that takes the raw data from the data broker, calculates the average over a given period of time, then uploads the result to the data broker for some other application to consume. This type of application is classified as a preprocessor as it allows us to perform some operation on the data before it is consumed elsewhere.

Device controller

Think of a machine that drills holes in a piece of material. When the machine is drilling we want to set a lamp to be green, when the machine is between drilling we want to set a lamp to be yellow and when the machine is powered off we want the light to be red. We can connect both the machine and the lamp to the Connectware, then read the status of the machine from the data broker into a service. This service can perform a check on the status and then write out to the data broker on a topic that controls the light to change colors. This type of application is classified as a Device controller as we are using input data to write to a device connected to the Connectware.

Data visualization

Think of an assembly line that outputs new products ready to be shipped. We can connect the sensors of the assembly to the Connectware and build a service that provides a web dashboard which outputs different graphs for the average output, speed, temperature and power used throughout the day. Administrators can then sign in to the Connectware and view this dashboard to see the status of the assembly line. This type of application is classified as a Data Visualization as it takes data from the Connectware and provides visualizations to allow easier consumption of important data.

Much more

These are only a few examples of the types of services that Cybus Connectware can deploy. As you will see below the Connectware is built on top of Docker, so as long as your application can be containerized it can be deployed on the Connectware.

How to create a Basic Service

Services are installed using a commissioning file. Within this text-based YAML file, all so-called resources like connections, endpoints, users, permissions and containerized applications a service uses are listed. Below you will find an example service commissioning file that deploys a Grafana instance which we will configure to show data provided by an OPC UA Server. The service will use one Cybus specific container to map MQTT to InfluxDB but the rest of the containers were taken from other projects and are easily integrated. If any section of the service commissioning file needs clarification please feel free to visit the Connectware Docs.

Description & Metadata

The first two sections of the service commissioning file give more general information. In the section description you can give a short description of the service which will also be displayed in the service’s details view on the Connectware Admin User Interface (Admin UI). The section metadata contains the meta information for the service that is being installed. Here we set a name of the service, version number, provide an icon for the thumbnail of the service, specify a provider and a related homepage.

#------------------------------------------------------------------------------
# CYBUS SERVICE COMMISSIONING
#------------------------------------------------------------------------------

description: |
  Service Commissioning File Example
  Cybus Learn - Service Basics
  https://learn.cybus.io/lessons/service-basics/

metadata:
  name: Service Basics Example
  version: 0.0.1
  icon: https://www.cybus.io/wp-content/uploads/2019/03/Cybus-logo-Claim-lang.svg
  provider: Cybus GmbH
  homepage: https://www.cybus.io

Parameters

To make a service configurable, we can specify parameters. Parameters are like referable variables whose values are defined by the user every time a service is to be installed or reconfigured. When applying a commissioning file in the Connectware, the user is asked to enter custom values for the parameters or to confirm the default values.

In this example we use parameters to make the OPC UA server address configurable for the case it would move to another location.

#------------------------------------------------------------------------------
# Parameters
#------------------------------------------------------------------------------
parameters:
    
  opcuaHost:
    type: string
    description: OPC UA Host Address
    default: opcuaserver.com

  opcuaPort:
    type: integer
    description: OPC UA Host Port
    default: 48010

Resources

In the Resources section we declare every resource that is needed for our service. All resources like connections, endpoints, users, permissions and containerized applications are configured here. The first resources we define are a user and and role granting permissions to our service.

#------------------------------------------------------------------------------
# Resources
#------------------------------------------------------------------------------

resources:
Users & Roles

To keep your infrastructure safe, some parts of the service have to use credentials for authorization on MQTT or HTTP interfaces. Within the commissioning file you can define users and roles which will be created on installation and can then be used by the service to access specific data or APIs.

A part of our service, which will access data via MQTT and forward it to InfluxDB, will have to authorize on the data interface. So we create the user dataMonitor with the global parameter Cybus::MqttPassword as a password, which holds an auto-generated, valid password for the Connectware’s internal MQTT broker, so we do not have to hardcode or manually configure it on installation. We assign the role dataReaderRole to it which is created afterwards and equipped with read permissions for any MQTT topic under building-automation (specified with the wildcard #).

To learn more about the details of creating users and roles check out the lesson on User Management.

  #----------------------------------------------------------------------------
  # USERS
  #----------------------------------------------------------------------------

  # purpose: Forwards data to InfluxDB and provides a Grafana instance
  dataMonitor:
    type: Cybus::User
    properties:
      password: !ref Cybus::MqttPassword
      roles:
        - !ref dataReaderRole
      permissions: []

  #----------------------------------------------------------------------------
  # ROLES
  #----------------------------------------------------------------------------

  # purpose: Reads data from specified MQTT topic
  dataReaderRole:
    type: Cybus::Role
    properties:
      permissions:
        - resource: 'building-automation/#'
          operation: read
          context: mqtt
Connections & Endpoints

In this lesson we will collect some simulated data from a public OPC UA server. We define a connection and an endpoint resource for this. For the connection we just need to specify the protocol as well as the server’s host address and port, which we define by referencing the previously declared parameters. For the endpoints we define three resources, each relating to a data source on the OPC UA server identified through the nodeId.

To learn more about the details of defining connections and endpoints utilizing various protocols, check out the other lessons on Cybus Learn. To learn more about the setup of an OPC UA connection and endpoints, check out the article How to connect and integrate an OPC UA server.

  #----------------------------------------------------------------------------
  # CONNECTIONS
  #----------------------------------------------------------------------------
      
  opcuaConnection:
    type: Cybus::Connection
    properties:
      protocol: Opcua
      connection:
        host: !ref opcuaHost
        port: !ref opcuaPort
        #username: myUsername
        #password: myPassword

  #----------------------------------------------------------------------------
  # ENDPOINTS
  #----------------------------------------------------------------------------
  
  Humidity:
    type: Cybus::Endpoint
    properties:
      protocol: Opcua
      connection: !ref opcuaConnection
      subscribe:
        nodeId: ns=3;s=AirConditioner_1.Humidity
        
  PowerConsumption:
    type: Cybus::Endpoint
    properties:
      protocol: Opcua
      connection: !ref opcuaConnection
      subscribe:
        nodeId: ns=3;s=AirConditioner_1.PowerConsumption
        
  Temperature:
    type: Cybus::Endpoint
    properties:
      protocol: Opcua
      connection: !ref opcuaConnection
      subscribe:
        nodeId: ns=3;s=AirConditioner_1.Temperature
Mappings

The data collected by our endpoints should be available to our service on the MQTT data interface provided by the internal Connectware MQTT Broker. The data is already available on service related MQTT topics (under services) but we need the data in a place which our newly created user can access (under building-automation). So we create a mapping that subscribes to our endpoints and publishes any received data to the desired MQTT topics.

For more information on the creation of mappings again check out the lesson How to connect and integrate an OPC UA server.

  #----------------------------------------------------------------------------
  # MAPPINGS
  #----------------------------------------------------------------------------
  
  mapping:
    type: Cybus::Mapping
    properties:
      mappings:
        - subscribe:
            endpoint: !ref Humidity
          publish:
            topic: 'building-automation/airconditioner/1/humidity'
        - subscribe:
            endpoint: !ref PowerConsumption
          publish:
            topic: 'building-automation/airconditioner/1/power-consumption'
        - subscribe:
            endpoint: !ref Temperature
          publish:
            topic: 'building-automation/airconditioner/1/temperature'
Volumes

A volume is a resource that represents a storage space and can be associated to containers. We want to utilize two containers, which will need additional storage space, so we create a volume for each of them.

  #----------------------------------------------------------------------------
  # VOLUMES
  #----------------------------------------------------------------------------

  grafanaVolume:
    type: Cybus::Volume

  influxdbVolume:
    type: Cybus::Volume
Frontends

In the frontends section we define the interfaces that the service will provide on a Connectware instance. Concerning the frontend we want to declare two resources. First we create an ingress route which will then point to a specific port in the service container. This allows services to provide web dashboards and REST APIs which can then be accessed through the HTTPS interface of the Cybus Connectware. In this case we define an HTTP interface and point it to port 3000 on the Grafana container which will allow us to access the dashboards. Second we define a link to the ingress route, which will simply provide a button named Dashboard on our service details view in the Connectware Admin UI. To learn more details about ingress route resources check out the Connectware Docs.

  #----------------------------------------------------------------------------
  # FRONTENDS
  #----------------------------------------------------------------------------

  # Grafana
  grafanaURL:
    type: Cybus::IngressRoute
    properties:
      container: !ref genericGrafana
      type: http
      slug: grafana
      target:
        path: '/'
        port: 3000

  dashboard:
    type: Cybus::Link
    properties:
      name: Dashboard
      ingressRoute: !ref grafanaURL
      href: ''
Containers

The containers section comprises the Docker Containers the service will run. These containers can either come from the Official Docker Registry or from the Cybus Registry. That means any application that is deployed on the Connectware can take full advantage of all the containerized software on Docker Hub and your custom containerized software delivered securely through the Cybus Registry. In the example below we pull the official Grafana and InfluxDB containers from Docker Hub and then use a custom Cybus container to map the MQTT data to InfluxDB. The different options that can be used when configuring these containers can be found in the Connectware Docs and concerning the container-specific environmental variables defined under the property environment you should check out the container’s documentation .

  #----------------------------------------------------------------------------
  # CONTAINERS
  #----------------------------------------------------------------------------

  influxdb:
    type: Cybus::Container
    properties:
      image: registry.cybus.io/cybus-services/influxdb:1.7.8-alpine
      volumes:
        - !sub '${influxdbVolume}:/var/lib/influxdb'
      environment:
        INFLUXDB_DB: generic

  genericGrafana:
    type: Cybus::Container
    properties:
      image: registry.cybus.io/cybus-services/generic-grafana:1.3.0
      volumes:
        - !sub '${grafanaVolume}:/var/lib/grafana'
      environment:
        GF_SERVER_ROOT_URL: !sub '/services/${Cybus::ServiceId}/grafana'
        GF_AUTH_ANONYMOUS_ENABLED: true
        INFLUX_HOST: !ref influxdb
        INFLUX_PORT: 8086
        INFLUX_DB: generic

  influxdbPush:
    type: Cybus::Container
    properties:
      image: registry.cybus.io/cybus-services/cybus-influxdb-push:2.0.1
      environment:
        MQTT_HOST: !ref Cybus::MqttHost
        MQTT_USERNAME: !sub '${Cybus::ServiceId}.dataMonitor'
        MQTT_PASSWORD: !ref Cybus::MqttPassword
        MQTT_SUBSCRIPTIONS: 'building-automation/#'
        INFLUXDB_HOST: !ref influxdb

How to install a Service

Now that we know what a service is and we have configured our own example we can install it on the Connectware. The service commissioning file can be found on GitHub.

Install Service

1) Open the Admin UI of your Connectware instance and navigate to the services section.

2) Click the plus-button in the upper right corner to add a service.

3) Select the service-example-commissioning-file.yml on your computer and click Install to confirm the default values and start the installation.

4) Click the now existing entry of your service in the list to open the details view.

5) Click the enable button on your Service.

Explore Service

Now that you have successfully installed and enabled a service we can use the provided dashboard and configure Grafana to visualize our simulation data.

1) First click the Dashboard button on the service detail view.

2) The welcome screen of Grafana has now opened in a new tab. Initially you need to log in to modify any dashboards. Click the Sign In button in the bottom left corner and log in with your Admin UI credentials.

3) As soon as you are logged in and back to the welcome screen go to the side menu and create a new dashboard.

4) Choose Add Query from the new panel.

5) Through service commissioning our InfluxDB is already connected to Grafana and selected as default data base. Click on select measurement and a list will present the data to you that we mapped on the MQTT topics.

6) With the four circles on the left side you can switch between different setting categories and for example edit the name of the panel.

7) In the upper right corner you can choose the time range and refresh rate. Next to this click the Save dashboard button, choose a name for the dashboard and click Save.

8) Clicking the highlighted Add panel button you can extend your dashboard with more panels like that.

The query editor for Grafana is incredibly powerful and can do much more then we will show in this tutorial. For more details read the documentation here.

Summary

In this lesson we learned what services are and made a basic Grafana service whose configuration gave a short demonstration of the various resources a service can utilize and how they are configured to create a consistent context of cooperation. We then configured Grafana to visualize humidity data from our simulation OPC UA server.

This article should have given you an overview over the possibilities that services offer in terms of interconnectivity. Knowing the methods of utilizing containers, including data sources, organizing their data flow and managing their access permissions should give you a hint of how you can structure the first own commissioning file for your Connectware!

Going further

To learn more about the Cybus Connectware check out our Connectware Docs or read more lessons here on Cybus Learn. To learn more on using the Grafana service read the Grafana Documentation.

Check out our media to this article: