Smart DIH Orchestration: Kubernetes

Overview

Kubernetes An open-source container orchestration system for automating software deployment, scaling, and management of containerized applications. is an open-source orchestration This is the automated configuration, management, and coordination of computer systems, applications, and services. Orchestration strings together multiple tasks in order to execute and easily manage a larger workflow or process. These processes can consist of multiple complex tasks that are automated and can involve multiple systems. Kubernetes, used by GigaSpaces, is a popular open source platform for container orchestration. system for automating the deployment, scaling and management of containerized applications.

Smart DIH Smart DIH allows enterprises to develop and deploy digital services in an agile manner, without disturbing core business applications. This is achieved by creating an event-driven, highly performing, efficient and available replica of the data from multiple systems and applications, (Digital Integration Hub) is at its core an encapsulation of multiple components that serve as an integration and acceleration of data streams.

On the outbound, Smart DIH Digital Integration Hub. An application architecture that decouples digital applications from the systems of record, and aggregates operational data into a low-latency data fabric. can be configured to connect to different components of the client such as databases, streams, APIs and applications.

To that end, our Smart DIH uses Kubernetes as its orchestration solution and as an enabler for using advanced technologies like memory grid, streaming and automation.

 

Using Kubernetes, our Smart DIH solution offers production grade qualities in the following areas:

1. Network and compute control

2. High Availability and Self-Healing

3. Security

4. TCO reduction and maintainability

Smart DIH installation using Kubernetes Helm charts makes the deployment of the solution easier for operational staff as best practices and standard Kubernetes procedures are followed. The installation is flexible and can be tailored to user needs by handling a rich set of deployment parameters.

 

Follow this guide for installing Smart DIH using Kubernetes

Network and Compute

Cloud Installation

Smart DIH can be installed on any of the major cloud vendors. Typically, start by setting a virtual network (i.e. VPC), then Kubernetes can be installed followed by Smart DIH. Connectivity to your current databases and applications can be established using appropriate gateways. Client applications can run within the same network, using a VPN connection or other means that adhere to security requirements (see security).

On-Premise Installations

Once Kubernetes is installed on-premise, be that a bare-metal installation or a private cloud, Smart DIH can be installed and connected to your existing applications.

 

Hybrid Installations

A hybrid installation typically involves two environments that differ in some characteristics.

An example could be on-premise (e.g. an air-gapped environment) and cloud environment, where the applications that consume the data are divided between the two (for security reasons for example). In such a case 2 Smart DIH installations would be required and data would be replicated (full or partial) between the two.

 

High Availability (HA) and Self-Healing

Space High Availability

The In-Memory Data Grid (IMDG In-Memory Data Grid. A simple to deploy, highly distributed, and cost-effective solution for accelerating and scaling services and applications. It is a high throughput and low latency data fabric that minimizes access to high-latency, hard-disk-drive-based or solid-state-drive-based data storage. The application and the data co-locate in the same memory space, reducing data movement over the network and providing both data and application scalability.) is at the center of Smart DIH. The data grid (also called a space) is maintained by an automation component (Kubernetes operator) that guarantees that the data will not be lost in the case of a pod going down.

Pods that serve the data grid, hold the data in memory and on disk (see Tiered Storage Automatically assigns data to different categories of storage types based on considerations of cost, performance, availability, and recovery. capabilities). The data grid is divided into partitions and each pod is assigned to hold a partition of the data as well as an availability mode - a “primary” or a “backup”. The Primary pods are online, however, if a primary pod goes down, the backup pod then becomes the primary and a new backup is quickly reassembled. This flow avoids downtime as data is seamlessly served from the pods memory by switching between a primary pod and its backup.

Partitions of the data grid are also backed up by the disk. This is a secondary measure to guarantee high availability. Availability for the data on disk (which can be larger than memory) is maintained by the storage facility and it is kept intact if the pod goes down.

 

In the diagram below, if POD-0 goes down POD-1 becomes the primary and Kubernetes will create a new POD-0 which will then be tagged as the backup.

Services High Availability

Low-code services are created by the user in order to open a channel for applications to interact with the data grid using a REST API REpresentational State Transfer. Application Programming Interface An API, or application programming interface, is a set of rules that define how applications or devices can connect to and communicate with each other. A REST API is an API that conforms to the design principles of the REST, or representational state transfer architectural style.. Due to this, it is important to maintain their high availability. These services are stateless and Smart DIH leverages the replication attribute of Kubernetes to maintain high availability.

In the diagram below, service “ABC” is replicated and serviced using a load balancer. In the case of one of the services going down the other continues to serve its client.

Security

Responsibilities

Smart DIH has been designed with strong security defaults in mind. These defaults include always-on authentication, authorization, encryption in transit and encryption at rest REpresentational State Transfer. Application Programming Interface An API, or application programming interface, is a set of rules that define how applications or devices can connect to and communicate with each other. A REST API is an API that conforms to the design principles of the REST, or representational state transfer architectural style. and no access from the Internet by default. Smart DIH includes the notion of roles that can be attached to users to preserve a minimal access policy.

Access

Authentication and authorization are at the core of the system. Access to every asset of the system is protected by these fundamentals. Smart DIH supports a rich set of authentication and authorization capabilities and can seamlessly connected with SSO Single sign-on (SSO) is an authentication scheme that allows a user to log in with a single ID to any of several related, yet independent, software systems. (single sign on) and various IDPs (IDentity Providers),

At Transit

Data is encrypted in transit. It is up to the user to open a public access and Gigaspaces will provide means to secure data at transit using protocols such as TLS Transport Layer Security, or TLS, is a widely adopted security protocol designed to facilitate privacy and data security for communications over the Internet. A primary use case of TLS is encrypting the communication between web applications and servers.. The typical connection is within the private network, This can be constructed using networking techniques such as VPN, Private link, VPC peering or other related protocols.

At Rest

Encryption can be set up by Smart DIH or by the customer to ensure that the data is secured at rest.

TCO Reduction and Maintainability

Installation

Smart DIH is installed with standard Kubernetes tools using common practices and flows. Helm is the go-to application used for installing Smart DIH and enables installation in single command using a Smart DIH Helm umbrella chart In Kubernetes, A helm umbrella chart is when a chart is created for the purpose of grouping together related subcharts/services, such as to compose a whole application or deployment. installation. A rich set of values can be applied upon installation, creating a customized deployment. Upgrades can leverage these tools so that future enhancements of Smart DIH can also occur with minimal effort.

Elasticity

Kubernetes has built-in mechanisms that expand and contract resources depending on load (given cloud support). This means that during run-time the system deployment is optimized for the current workload.

Control

Kubernetes tools support various operations for managing the system. This includes control resources count, security, metrics and logging parameters. Smart DIH in this sense provides additional controls at the Smart DIH level (SpaceDeck) which may also affect the underlying deployment by leveraging Kubernetes operators.