Smart DIH - Platform Components

Smart DIHClosed is an operational data hub designed to address IT challenges of supporting modern, digital applications over heterogeneous, mostly legacy data architectures. By creating an event-driven, highly performing, efficient and available replica of the data from multiple systems and applications, Smart DIHClosed allows enterprises to develop and deploy digital services in an agile manner, without disturbing core business applications. Strategic initiatives such as integration data hub, digital innovation over legacy systems, API scaling, cloud migration, and business 360 are common use cases for which Smart DIH is utilized.

Platform Components

At a high level, GigaSpaces Smart DIH platform bridges between Enterprise’s data sources and its high-end applications. It does so by streaming the data through these 3 stages (south to north):

  1. Data Capture and Transformation

  2. Caching and Backup

  3. Data Servicing

Another aspect is the platform’s control and monitoring facilities. More information about this can be found in the Application Lifecycle page.

 

Data Integration

  1. System of Record Agent

    This thin agent which resides close to the System of Record (e.g., DB2), fetches raw data changes from the data source and sends it over to the transformation stage.

  2. Change Data Capture (CDCClosed) Technology for Real-time Events

    CDC tools are implemented to capture real-time changes from the on-premise System of Record. These tools capture and propagate changes to downstream systems.

  3. KafkaClosed Bus for Event Sourcing

    Apache Kafka is used as a message bus to stream data change events from the agent to the data hub. Kafka ensures reliable and scalable data streaming.

  4. Data Catalog

    The DIH platform learns the source's data structures and creates a catalog where the source and data-grid metadata is kept.

  5. Data Transformation

    A layer that extracts and transforms the incoming data into an effective data structure. Multiple on-line functions can be applied on the stream..

Caching and Backup

  1. GigaSpaces Data Hub

    An in-memory data grid (IMDGClosed) for high-performance caching. It provides fast access to frequently accessed data and enhances the system's overall responsiveness.

  2. Tiered StorageClosed

    Through Intelligent Data Tiering, data can extend beyond the data-grid. The data grid oversees the persistence of data using a rule-based process.

Data Services

  1. Micro-Services

    Using Smart DIH, micro-services or server-less functions can be created be consuming data from data grid. These services cover specific business logic, providing a scalable and modular architecture. Refer to SpaceDeck – Services to see how micro-services can be created through SpaceDeck.

  2. API Gateway

    API Gateway Integration (e.g., AWS API Gateway, Azure API Management) manages and exposes APIs for data access and integration. The API Gateway can enforce security policies, rate limiting, and handle authentication.

Extensions

The following enhancements can be tailored to the needs of our customers:

  1. MirrorClosed to Persistence Database

    With the mirror customization functionality a cloud-based data-store (e.g., Amazon RDS, Azure Database for PostgreSQL) can be used to replicate data persistently. This database serves as a replica of the on-premise System of Record.

  2. Event Driven Application Push

    GigaSpaces data-grid technology has the ability to notify the application in real time when there is any change to the data. To that end, the customer can add a program with the behavior and destination using special processing unitsClosed.

  3. Network Data Sync

    Apache Kafka is used as a message bus to stream data change events from the agent to the data hub. Kafka ensures reliable and scalable data streaming.

 

For information about Smart DIH refer back to the Smart DIH contents page and choose another topic.