Smart DIH - Application Life Cycle & Performance

Application Life Cycle

The following diagram represents the Smart DIHClosed platform life Cycle:

Initial Setup

The following steps are required in order to install the Smart DIHClosed system

  1. Hardware and Network Setup - Either on-premise or cloud

    1. Network Configuration - Virtual Network, Routers, VPN

    2. Compute Resources - including cpu/memory, Disk

    3. Security Setup - Firewalls, Gateways, access rights

  2. SORClosed Agent Installation - File and database configuration

  3. KubernetesClosed and Application Installation - This is where the controller, data-grid and streaming (KafkaClosed) are provisioned

    For installation instructions refer to our Smart DIH Kubernetes Installation Guide

  4. In SpaceDeckClosed

    1. Configure the IDPClosed, Roles and Service Accounts (performed by the Admin)

    2. Setup Data Sources - point to the SOR agent and setup connection configuration

    3. Set up Spaces - define the data-grid spaces for holding the data

    4. Set up Pipelines - define the data that goes into the data-grid

    5. Set up Services - define the micro-service/s that consume the data from the data-grid (either by SQL or Code) and the APIs

    Refer to our SpaceDeck User Guide for comprehensive information.

Control, Logging & Monitoring

Control of Smart DIH is managed through SpaceDeck. Platform logs can also be accessed via SpaceDeck. For more advanced logging queries and ELK stack can be attached to the pods (components) logs. And example of such a service is Amazon OpenSearch.

Monitoring can be performed by both utilizing the tools that integrated within SpaceDeck and using dashboards in GrafanaClosed. Upon startup a few dashboards are uploaded to Grafana including data-grid metrics.

Refer to our SpaceDeck User Guide for comprehensive information.

Updates & Upgrades

The user can update the Smart DIH platform in the following ways:

  1. Add Spaces

  2. Update/Add pipelines

  3. Update/Add services

  4. Upgrade Kubernetes

  5. Upgrade the platform

Refer to our SpaceDeck User Guide and Smart DIH Kubernetes Installation Guide for additional information.

Reliability

Many of the Smart DIH components contain the appropriate redundancy and are spread across multiple availability zones. The following components maintain this ability:

  1. Data-Grid - Special primary/backup technology maintains high availability of the cached data. The Kubernetes nodes are setup with anti-affinityClosed mode for the primary and backup copies.

  2. ZookeeperClosed - Zookeeper that holds some configurations needs a quorum of instances that span over multiple zones.

  3. Kafka - Kafka is configured by default with a replication factor higher than one. So there are always additional copies of the data in multiple zones.

  4. FlinkClosed (in the transformation layer)- Launching a few instances of the task manager ensures that tasks can be run continuously.

Performance

GigaSpaces IMDGClosed (in-memory data grid) is a technology that manages data queries from the data-grid. Read and writes queries are executed in a few milliseconds (depending on the complexity). Due to this, the services accessing the data-grid can benefit from its high performance.

 

For information about Smart DIH refer back to the Smart DIH contents page and choose another topic.