Smart DIH - Application Life Cycle & Performance
Application Life Cycle
The following diagram represents the Smart DIH platform life Cycle:
Initial Setup
The following steps are required in order to install the Smart DIH system
-
Hardware and Network Setup - Either on-premise or cloud
-
Network Configuration - Virtual Network, Routers, VPN
-
Compute Resources - including cpu/memory, Disk
-
Security Setup - Firewalls, Gateways, access rights
-
-
SOR
Agent Installation - File and database configuration
-
Kubernetes
and Application Installation - This is where the controller, data-grid and streaming (Kafka
) are provisioned
For installation instructions refer to our Smart DIH Kubernetes Installation Guide
-
In SpaceDeck
-
Configure the IDP
, Roles and Service Accounts (performed by the Admin)
-
Setup Data Sources - point to the SOR agent and setup connection configuration
-
Set up Spaces - define the data-grid spaces for holding the data
-
Set up Pipelines - define the data that goes into the data-grid
-
Set up Services - define the micro-service/s that consume the data from the data-grid (either by SQL or Code) and the APIs
Refer to our SpaceDeck User Guide for comprehensive information.
-
Control, Logging & Monitoring
Control of Smart DIH is managed through SpaceDeck. Platform logs can also be accessed via SpaceDeck. For more advanced logging queries and ELK stack can be attached to the pods (components) logs. And example of such a service is Amazon OpenSearch.
Monitoring can be performed by both utilizing the tools that integrated within SpaceDeck and using dashboards in Grafana. Upon startup a few dashboards are uploaded to Grafana including data-grid metrics.
Refer to our SpaceDeck User Guide for comprehensive information.
Updates & Upgrades
The user can update the Smart DIH platform in the following ways:
-
Add Spaces
-
Update/Add pipelines
-
Update/Add services
-
Upgrade Kubernetes
-
Upgrade the platform
Refer to our SpaceDeck User Guide and Smart DIH Kubernetes Installation Guide for additional information.
Reliability
Many of the Smart DIH components contain the appropriate redundancy and are spread across multiple availability zones. The following components maintain this ability:
-
Data-Grid - Special primary/backup technology maintains high availability of the cached data. The Kubernetes nodes are setup with anti-affinity
mode for the primary and backup copies.
-
Zookeeper
- Zookeeper that holds some configurations needs a quorum of instances that span over multiple zones.
-
Kafka - Kafka is configured by default with a replication factor higher than one. So there are always additional copies of the data in multiple zones.
-
Flink
(in the transformation layer)- Launching a few instances of the task manager ensures that tasks can be run continuously.
Performance
GigaSpaces IMDG (in-memory data grid) is a technology that manages data queries from the data-grid. Read and writes queries are executed in a few milliseconds (depending on the complexity). Due to this, the services accessing the data-grid can benefit from its high performance.
For information about Smart DIH refer back to the Smart DIH contents page and choose another topic.