InsightEdge Platform with XAP Core
In-Memory Data Grid
GigaSpaces XAP is the core in-memory data grid for the InsightEdge Platform. XAP is designed to support ultra-low latency and high-throughput applications that follow Space-Based Architecture design patterns. XAP is based on GigaSpaces’ Spring-based OpenSpaces Framework as the primary development environment, and uses the GigaSpaces space-based runtime to deliver core middleware facilities: messaging, data caching and parallelization.
The following is a brief overview of some XAP data grid capabilities.
Memory and Storage
Data Replication and Persistence
Applications running on XAP can be scaled out linearly, because XAP uses the Space-Based Architecture (SBA) as a primary design pattern. With SBA, applications are built out of a set of self-sufficient units, known as Processing Units (PU). These units are completely independent of each other, so that the application can scale indefinitely without increasing complexity, just by adding more units. SBA is based on the Tuple Space paradigm; it follows many of the principles of Service-Oriented Architecture and Event-Driven Architecture , as well as elements of grid computing .
Applications and Microservices
Applications deployed on XAP are very much like JEE distributed server-side applications. Each application is composed of one or more scalable modules, and contains one or more business logic code components, domain model objects (entries), in-memory data storage, messaging, and event processing code.
A module in SBA is called a Processing Unit. This is the smallest unit for cross-cluster scaling. From a technical standpoint, a Processing Unit is a collection of classes and a configuration file, packaged as a single unit of scale.
- Java Processing Unit - has code components in Java, and may contain a space component.
- .NET Processing Unit - has its code components in .NET, and may contain a space component.
- Web Processing Unit - contains web applications. This Processing Unit can be packaged as a standard JEE WAR (Web Archive) file, and may contain a space component.
A Processing Unit package structure is defined, based on the technology:
- A Java Processing Unit is packaged usually as a JAR file, modeled after the Spring DI structure. The package can also be within a file structure based on the same structure.
- A .NET Processing Unit is packaged as a directory structure. For more information, refer to the XAP.NET Guide.
- A Web Processing Unit is packaged as a standard JEE WAR file.
The services will usually operate on data that is stored within the same space partition, providing memory access within the same process address space. This mode of interaction allows the minimal latency possible, as data is accessed by reference, as opposed to serialization required for out-of-process communication.
Processing units can be deployed through the GigaSpaces Management Center, Web Management Console, Command Line Interface and REST Manager API.
Analytics and Big Data
The InsightEdge platform provides a first-class integration between Apache Spark and any data grid capability. This allows hybrid/transactional analytics processing by co-locating Spark jobs in-place with low-latency data grid applications. The platform includes a full Spark distribution that is managed by a highly-available clustering tier that provides auto-healing, as well as local and geographical redundancy. The following is a sample of available InsightEdge analytics capabilities:
- Interactive Data Analytics Web UI
- Spark ETL and Transformations
- Spark SQL
- Machine Learning
- Geospatial Analytics
- Python API
Distributed SQL and Visualization (BI)
The InsightEdge Enterprise edition provides a SQL-99 query engine for
read-only operations. The purpose of this query engine is to provide business intelligence developers and grid administrators with a mechanism to perform interactive data analysis through vizualization tools (such as: Tableau, QlikView and MicroStrategy).
All components and applications in the platform are managed, deployed, and monitored through a highly available, resilient service discovery that comprises the XAP service grid and Apache ZooKeeper . The combination is responsible for system health monitoring, service discovery, and resource orchestration, as well as other low-level services for scaling your workloads across your data center or cloud.
In addition to the management center, and web and command line interfaces, GigaSpaces provides a grid management API interface exposed through the standard HTTP REST protocol. This REST interface can be called from your custom management and administration scripts to support different deployment and provisioning lifecycle operations. In addition, the REST Manager API provides a plug-in provider to implement your own orchestration/integration endpoints.
Management and Monitoring
GigaSpaces provides several components for monitoring and troubleshooting your XAP or InsightEdge Enterprise deployments. These include:
- GigaSpaces Management Center
- Web Management Console
- Command Line Interface
- Metrics Reporting and Integration
- Administrative Alerts
Security and Auditing
GigaSpaces provides security features that allow the administrators to implement security controls across any component. Security management in GigaSpaces consists of security controls to access the entire platform stack, from SSL encryption of data traffic to role-based authentication/authorization for deployment, data access and provisioning.