GigaSpaces Smart ODS

GigaSpaces Smart ODS is part of the GigaSpaces Portfolio — Smart Cache, Smart ODS and Smart Augmented Transactions.

InsightEdge Smart ODS is a distributed in-memory Digital Integration Hub (DIH), with unparalleled low-latency, high-performance and scale that aggregates and offloads from multiple back-end systems of record and data stores on premise and on cloud. With a unified API layer, digital applications can be decoupled from disparate systems of record to increase agility and ensure always-on services.

When an existing Operational Data Store is already deployed, Smart ODS can augment it to gain better performance, indexing, data integration and ETL replacement. It can be done by either mirroring the ODS actions to the data stores, or by using Smart ODS’s built-in Change Data Capture (CDC) that connects directly with your systems of record, non-intrusively (agentless).

Smart ODS System Overview

The Smart ODS environment can be illustrated as follows:

Architecture of Smart ODS

1. One-Click Connect to Data Sources

 

Connection to the data source can be either by mirroring the ODS actions to the data store, or by using Smart ODS’s built-in Change Data Capture (CDC) that connects directly with the systems of record, non-intrusively (agentless). Data ingestion from the CDC is in millisecond latency and keeps the ODS always in sync with all the different data sources. This is delivered as part of Smart ODS with unified installation and orchestration.

2. Automated Data Tiering

Data is stored in multiple data storage tiers — hot (in-memory), warm (SSD), and cold (disk/external data store)

3. High Availability and Remote Replication

InsideEdge Smart ODS is an enterprise-grade product that provides 5-nines availability. The platform is highly available by having primary and backup instances of data partitions.

Additionally, the platform provides for tier storage and disk persistency, with built-in mirroring service to external databases including initial load and store to the database.

In addition, InsightEdge provides efficient data replication between sites (Active-Passive/Active-Active) including hybrid environments.

 

InsightEdge allows users to utilize rich support for declarative transaction management. The declarative transaction support can be easily utilized with the GigaSpace Interface.

4. High Performance Microservices Framework

GigaSpaces supports Task execution in an asynchronous manner, co-located with the Space (Processing Unit that started an embedded Space). Tasks can be executed directly on a specific cluster member using routing declarations. Tasks can also be executed in “broadcast” mode on all the primary cluster members concurrently, and reduced to a single result on the client side.

A Distributed Task a Tasks that is executed more than once (concurrently). It returns a result that is the reduced product of all operations. This reduction is calculated in the Task’s reduce method.

Phase 1 - Sending the Tasks to be executed:

Phase 2 - Getting the results back to be reduced:

Co-locating data and code significantly improves application performance by reducing serialization and networking overhead. Data processing is performed locally by direct memory access to the data in the server memory space.

The InsightEdge platform provides a unified capability for distributed data and processing, which enables it to easily scale up and out, using a single API.

Having a unified processing and data storage platform encapsulates all of the required features for keeping the data and processing highly available under the same API framework. In addition, a fast remote replication facility is provided out of the box, including encryption, compression, and filtering or other custom requirements.

5. Event Driven Architecture

InsightEdge ODS provides the user with a rich API to capture of real-time events and data streams, to automatically trigger notifications, based on defined business logic. Digital apps and external business processes can easily subscribe to any event in pull or push mode without impact on performance.

6. APIs

Smart ODS supports the full SQL-99 standard, allowing complex queries to be performed with optimal speed and minimal memory footprint. Standard SQL requests used for relational data manipulation can be used without requiring source code changes.

Using Java and .NET APIs, advanced capabilities are available — such as object store, and various data modeling options.

InsideEdge features native integration with Kafka to consume and produce data to and from Kafka topics.

REST-API and command line interfaces are also available.

Smart ODS can easily integrate with BI tools such as Tableau, Looker, and PowerBI.

Deploy Anywhere

Click here to get started with Smart ODS.