In-Memory Data Grid - achieve unparalleled speed, persistence, and accuracy.SpaceDeck – Data Pipelines Overview
Data Pipelines are flexible, configurable, no-code components that gather data from a mix of System of Record databases and tables. This data is loaded into the extremely high speed GigaSpaces data grid, where it can be accessed on a read/write basis.
Security Roles & Permissions
SpaceDeck
GigaSpaces intuitive, streamlined user interface to set up, manage and control their environment. Using SpaceDeck, users can define the tools to bring legacy System of Record (SoR) databases into the in-memory data grid that is the core of the GigaSpaces system. is now secure. Refer to Security Overview for details about the security privileges that are required for the various Pipeline related activities.
Data Pipelines - Main Screen
The Data Pipeline A series of data processing steps, including extraction, transformation, and loading (ETL), that move data from its source to a destination system. Data pipelines are essential for integrating and managing data flows. main screen shows the status of the existing data pipelines. You can see details of a pipeline by clicking on its name in the list.
Data Pipelines - Main Screen - Fields
Field Names:
-
Export - Click this to export selected rows in the table.
-
Group by Status – toggle to show the pipelines grouped status
-
Create New + – click this to configure a new pipeline. See SpaceDeck – Data Pipeline – Create New Pipeline for instructions on how to define a new pipeline.
-
NAME – name assigned to the pipeline
The Pipeline name has no naming limitations
-
STATUS – one of the following conditions: started/stopped/not starting/restarting/not found
-
TYPE – the connection type for the pipeline, for example, IIDR
IBM Infosphere Data Replication.
This is a solution to efficiently capture and replicate data, and changes made to the data in real-time from various data sources, including mainframes, and streams them to target systems. For example, used to move data from databases to the In-Memory Data Grid. It is used for Continuous Data Capture (CDC) to keep data synchronized across environments. -
ORIGIN – data source type from the System of Record, e.g. Oracle
-
DESTINATION SPACE- the Space
Where GigaSpaces data is stored. It is the logical cache that holds data objects in memory and might also hold them in layered in tiering. Data is hosted from multiple SoRs, consolidated as a unified data model. that has been defined -
SUBSCRIPTION NAME -
-
TOTAL OPERATIONS (LAST 24 HOURS) – number of operations performed by the pipeline in the last 24 hours
-
TOTAL ERRORS (LAST 24 HOURS) – number of error operations performed by the pipeline in the last 24 hours
- Search It is also possible to filter a search. Once the filters are set, click Apply:
-
Once a Pipeline has been defined, the kebab menu on the far right can be used to either Start or Delete a pipeline
For more information: