GigaSpaces Smart Cache
InsightEdge Smart Cache is part of the InsightEdge Portfolio — Smart Cache and Smart DIH.
Smart Cache provides fast, scalable cache from any data store. It is fully SQL-99 compliant, and can connect in one click to any operational data store. Additionally it provides out-of-the-box advanced persistency options.
Smart Cache Overview
Architecture of Smart Cache
1. Connect to Data Source
GigaSpaces Data Integration is a simple, reusable way to create Data Pipelines that allow access to application data without requiring user code. See Data Integration for details.
GigaSpaces provides a simple wizard interface to connect to any multiple relational data sources. First the user selects either a database or data warehouse, defines various properties of the data source and the naming of the space and stateful service at InsightEdge, then the platform automatically scans the data source and presents it for the user to review the data structure, choose the required data to load, with the option to rename tables, columns and change indexes. Then the user can choose between two options: Download the generated project files, customize as required and deploy or automatically deploy on the cluster. For more information click here.
One-Click connect interface
2. Automated Data Tiering
Data is stored in multiple data storage tiers — hot (in-memory), warm (SSD), and cold (disk/external data store)
User-defined business rules allows you to place the most important data in hot storage, while other data can be in warm or cold storage.
InsightEdge manages the full data lifecycle by automatically storing the relevant data in the relevant tiers, federating the queries under the hood to gather all the data in optimal latency and evicting the data according to various policies, from least-recently-used (LRU) to user custom policies. Click here for more information about tiered storage.
"Important" does not necessarily mean "most frequently accessed" — for example, your business rules may place premium customers in hot storage, even if their data is less frequently required.
3. Optimized Performance
Smart Cache provides unlimited amount of secondary indexes for the stored data, such as Exact, Range, Collections, Compound, Text, and Geo Spatial for optimal performance.
Unlike key-value stores that need to duplicate the entire data store for each defined index, InsightEdge only creates additional indexing data structures on one instance of data.
Smart Cache advanced indexing
Perform an aggregation activity across data stored within space Aggregators are executed by iterating the internal data grid structure that maintains the space objects Out of the box aggregators such as SUM, AVG, MIN, GROUP BY, etc. as well as CUSTOM AGGREGATION LOGIC Fast scan - No materialization of the original user data grid object when performing this iteration
Distributed server-side data aggregation and code co-location provides high concurrency, full data integrity and reduces networking and serialization overhead.
When the objects are written to the Data Grid, they are routed to the proper partition, according to a predefined attribute in the object that acts as the routing index.
User-defined rules can automatically scale up/down or scale out/in when system demands change.
For example a rule could state:
When the RAM in use exceeds 80% of the total currently allocated RAM for more than 2 minutes, scale up the RAM by a factor of 4.
Rules can also be time-dependent, for example automatically changing storage allocations before planned peak system workload.
Smart Cache APIs
Smart Cache supports the full SQL-99 standard, allowing complex queries to be performed with optimal speed and minimal memory footprint. Standard SQL requests used for relational data manipulation can be used without requiring source code changes.
Using Java and .NET APIs, advanced capabilities are available — such as object store, and various data modeling options.
Smart Cache can easily integrate with BI tools such as Tableau, Looker, and PowerBI.
Deployment options include:
Hot rolling deployment, upgrade, monitoring, scaling and recovery, all with no downtime
Support of Kubernetes and OpenShift orchestration systems
Multiple deployment options of Smart Cache