Data Integration

Data Integration allows data to be loaded into the GigaSpaces data grid using easily-defined, reusable Data Pipelines that require no user code.

A metrics dashboard shows data activity, including data volumes and statistics and any data validation errors or other access errors.

Information Required to Define a Data Pipeline

In order to create a new data pipeline, the following information is required:

• The name of the data pipeline, type of connector, URL of the data source, and user/password or other authentication data. This information identifies where the data flow is located.

• Selection of which tables and fields to use in the data source

• The name of the target Gigaspace Space that will access the data.

• Any validations to be performed on the source data, with stipulations of what to do if data errors occur.

• Definition of optional Tiered Storage and Storage Optimization features

• Actions to be performed if errors occur in the data loading process or if the target Space is redeployed.

Procedure to Define a Data Pipeline

A Data Pipeline can be created using the SpaceDeck User Interface. See SpaceDeck – Data Pipeline – Create New Pipeline.