Data Integration allows data to be loaded into the GigaSpaces data grid using easily-defined, reusable Data Pipelines that require no user code.
A metrics dashboard shows data activity, including data volumes and statistics and any data validation errors or other access errors.
Information Required to Define a Data Pipeline
In order to create a new data pipeline, the following information is required:
The name of the data pipeline, type of connector, URL of the data source, and user/password or other authentication data. This information identifies where the data flow is located.
Selection of which tables and fields to use in the data source
The name of the target Gigaspace Space that will access the data.
Any validations to be performed on the source data, with stipulations of what to do if data errors occur.
Definition of optional Tiered Storage and Storage Optimization features
Actions to be performed if errors occur in the data loading process or if the target Space is redeployed.
Procedure to Define a Data Pipeline
A Data Pipeline can be created using the SpaceDeck User Interface. See SpaceDeck – Data Pipeline – Create New Pipeline.