XAP

Using Grafana for Monitoring and Analytics

Overview

Monitoring a production environment of any system requires inspecting many statistics. GigaSpaces provides the Ops Manager for general monitoring and troubleshooting, and the Web Management Console provides various metrics (by polling the system's components) with the following limitations:

To overcome these limitations, GigaSpaces also provides a powerful and versatile framework for reporting metrics. Each component can report its own predefined or user-defined metrics to any database or tool by implementing a metrics reporter. In addition, GigaSpaces provides a built-in reporter for InfluxDB and integration with GrafanaClosed Grafana is a multi-platform open source analytics and interactive visualization web application. It provides charts, graphs, and alerts for the web when connected to supported data sources. to provide metrics storage and visualization:

  • InfluxDB is an open-source distributed time-series database with a powerful query language.
  • Grafana is an enterprise tool for organizations with a graph and dashboard builder for visualizing time-series metrics, which supports InfluxDB (and other data sources).

When these metrics are enabled, GigaSpaces reports them to InfluxDB, and the Web Management Console provides an embedded Grafana view in the Monitoring tab, as well as automatic creation of dashboards in Grafana whenever a new Processing UnitClosed This is the unit of packaging and deployment in the GigaSpaces Data Grid, and is essentially the main GigaSpaces service. The Processing Unit (PU) itself is typically deployed onto the Service Grid. When a Processing Unit is deployed, a Processing Unit instance is the actual runtime entity. is deployed.

InfluxDB and Grafana are both open-source and free, but are not bundled in GigaSpaces distribution. Installation is straightforward, as described below. Note that GigaSpaces's default metrics configuration is set to match the default settings of InfluxDB and Grafana, so if this is your first time we recommend leaving the default configuration to simplify the process.

Installing and Configuring InfluxDB

To install InfluxDB (1.8.0 or earlier), download from here and follow the installation instructions.

To configure InfluxDB, edit the metrics.xml file, found under$GS_HOME/charts/xap-manager/config for KubernetesClosed An open-source container orchestration system for automating software deployment, scaling, and management of containerized applications., and under $GS_HOME/config/metrics for ServiceGrid. Change the following part according to your InfluxDB host (myhost) and database name (mydb) that stores metrics:


    <grafana>
        <datasources>
            <datasource name="influxdb">
                <property name="type" value="influxdb"/>
                <property name="url" value="http://myhost:8086/db/mydb"/>
                <property name="username" value="root"/>
                <property name="password" value="root"/>
            </datasource>
            <datasource name="grafana">
                <property name="type" value="influxdb"/>
                <property name="url" value="http://myhost:8086/db/grafana"/>
                <property name="username" value="root"/>
                <property name="password" value="root"/>
                <property name="grafanaDB" value="true"/>
            </datasource>
        </datasources>
    </grafana>

Installing and Configuring Grafana

To install Grafana (6.7.3 or earlier), download from here, and follow the installation instructions.

Starting with Grafana version 6.2.0, the following flag in the defaults.ini file should be set to true in order to allow opening grafana within our web-ui:
allow_embedding=true

After installation, you need to configure Grafana to work with GigaSpaces's metrics configuration, which is located at $GS_HOME/charts/xap-manager/config/metrics.xml for Kubernetes, and at $GS_HOME/config/metrics/metrics.xml for ServiceGrid. Assuming you've installed InfluxDB and Grafana without changing the defaults, all you need to do is uncomment the InfluxDB reporter and grafana elements, as shown below:

<metrics-configuration>
    <reporters>
        <reporter name="influxdb">
            <property name="host" value="localhost"/>
            <property name="database" value="mydb"/>
        </reporter>
    </reporters>
    <grafana url="http://localhost:3000" api-key="" user="admin" password="admin">
        <datasources>
            <datasource name="xap">
                <property name="type" value="influxdb"/>
                <property name="isDefault" value="true"/>
                <property name="url" value="http://localhost:8086"/>
                <property name="database" value="mydb"/>
                <property name="access" value="proxy"/>
            </datasource>
        </datasources>
    </grafana>
</metrics-configuration>
  • reporter - InfluxDB installation automatically creates a database called mydb, which matches the default. See InfluxDB Reporter for information about other optional settings.
  • grafana - Grafana installation defaults to port 3000 with username/password admin/admin. In production you'll probably change those in Grafana, so make sure to change them in metrics.xml as well. Even better, you can generate an API key in Grafana and place it in the api-key attribute.
  • datasource - This tells Grafana how to connect to your InfluxDB database. Note that InfluxDB uses port 8086 by default for an API endpoint, and that we're connecting to the default mydb database.

We recommend verifying that the default database exists. Some InfluxDB packages do not automatically create the default mydb database. You can use the InfluxDB Web Admin or shell to check if the database exists, and create it if needed. For more information, see the Getting Started with InfluxDB topic on the InfluxDB website .

Getting Started

After you install InfluxDB and Grafana and configure the metrics.xml file, start the Web Management Console and navigate to the Monitoring tab - you'll see Grafana's home page (you'll probably get a login page on the first time - just type in the default admin/admin, and you'll get the home page.

Dashboards

Following are examples of predefined dashboards.

GigaSpaces Spaces Dashboard

image

GigaSpaces Infrastructure

image

How to Disable Grafana Metrics

By default, Grafana metrics are automatically enabled during product installation.

If you wish to disable Grafana metrics, proceed as follows: