Detailed Configuration: XAP Skyline

This topic contains detailed customization information required for the installation of XAP Skyline A highly customizable developer platform that allows building scalable HA with high throughput and ultra-low latency Java applications running on Kubernetes clusters

Helm values can be applied in the following two places:

1. Initial deployment of XAP GigaSpaces eXtreme Application Platform. Provides a powerful solution for data processing, launching, and running digital services Skyline. This is the umbrella installation. Refer to XAP Skyline Installation for comprehensive instructions.

2. Deploying a Space Where GigaSpaces data is stored. It is the logical cache that holds data objects in memory and might also hold them in layered in tiering. Data is hosted from multiple SoRs, consolidated as a unified data model. (or PU This is the unit of packaging and deployment in the GigaSpaces Data Grid, and is essentially the main GigaSpaces service. The Processing Unit (PU) itself is typically deployed onto the Service Grid. When a Processing Unit is deployed, a Processing Unit instance is the actual runtime entity.) when you change the Space deployment in order to add features such as Tiered Storage Automatically assigns data to different categories of storage types based on considerations of cost, performance, availability, and recovery., Resources and Probes. Refer to Deploying a Customer Processing Unit for more information. When the parameter belongs to the Space installation it is defined as a PU Parameter.

Setup

Feature Flags

Default values are shown in bold.

Helm Parameter	Values	Purpose
metrics.enabled	true/false	Adding influx and grafana deployments
manager.metrics.enabled	true/false	Manager generate metrics in influxdb
manager.securityService.enabled	true/false	Manager runs in secure mode (see security documentation)
manager.securityService.ingress.enabled	true/false	Adding ingress for security service access (e.g. for oauth2 handshake)
spacedeck.auth.enabled	true/false	SpaceDeck GigaSpaces intuitive, streamlined user interface to set up, manage and control their environment. Using SpaceDeck, users can define the tools to bring legacy System of Record (SoR) databases into the in-memory data grid that is the core of the GigaSpaces system. authentication enabled
manager.antiaffinity.enabled	true/false	Smart Cache Manager anti-affinity This describes the relationship between VMs and hosts or when related to Kubernetes between pods. Anti-Affinity will keep VM and hosts separated and using Kubernetes an anti-affinity rule tells the scheduler not to place the new pod on the same node if the label on the new pod matches the label on another pod. In this case, anti-affinity allows you to keep pods away from each other. It also allows you to prevent pods of a particular service from scheduling on the same nodes as pods of another service that are known to interfere with the performance of the pods of the first service.
manager.persistence.enabled	true/false	Enable space persistency (Tiered Storage)
manager.service.lrmi.enabled	true/false	Ability to access space with lrmi protocol from external hosts
operator.keystore.enabled	true/false	Ability to inject admission controller certificate using a volume
operator.autoCleanup.enabled	true/false	Clean up PU’s when operator is uninstalled
operator.statefulSetExtension.enabled	true/false	Enable adding more objects to a PU (see statefulset extensions)
dgw.enabled	true/false	Adding data gateway for direct JDBC Java DataBase Connectivity. This is an application programming interface (API) for the Java programming language, which defines how a client may access a database. driver access

Pods/Service/Ingress (Networking Configuration, H/A)

Network Configuration

By default, all services are configured with the ClusterIP. The following services have to be accessed from outside the cluster:

1. SpaceDeck - Used for running the UI.

2. XAP Security Service - For OAuth2.0 re-direction endpoint.

3. Data Gateway - When enabled, provides JDBC access (TCP connection).

The table below lists the details of the services. Nginx configuration is included for reference.

Service Name and Port	Type	Nginx Config ( if used)
SpaceDeck: 3000	http	Using ingress manifest
xap-security-service :9000	http	Using ingress manifest
xap-dgw-service: 5432	tcp	Using TCP mapping

Any of the services listed above can be modified to LoadBalancer, if it is supported by the Kubernetes An open-source container orchestration system for automating software deployment, scaling, and management of containerized applications. cluster. Alternatively, for the http services listed, an ingress can be used. For SpaceDeck and the XAP Security Service these ingress manifests are installed with the Smart DIH Smart DIH allows enterprises to develop and deploy digital services in an agile manner, without disturbing core business applications. This is achieved by creating an event-driven, highly performing, efficient and available replica of the data from multiple systems and applications, Helm installation by default.

Nginx Configuration

The Nginx controller can be used as a router for incoming traffic. As stated previously, another ingress controller or a Load Balancer can be used instead.

The controller is used for satisfying the following requirements:

TLS Transport Layer Security, or TLS, is a widely adopted security protocol designed to facilitate privacy and data security for communications over the Internet. A primary use case of TLS is encrypting the communication between web applications and servers. termination (this can be done outside as well)

Routing The mechanism that is in charge of routing the objects into and out of the corresponding partitions. The routing is based on a designated attribute inside the objects that are written to the Space, called the Routing Index. according to Kubernetes ingress declarations for HTTP traffic

TCP routing by setting TCP mapping in the Nginx configuration (using configmap). Refer to this Kubernetes guide for specific TCP configuration.

Nginx Deployment Example

In the example value file to install nginx below, a single pod Nginx installation using a hostPort connection is shown. Replace the parameters according to the list below.

nginx-values.yaml

controller:
   ingressClassByName: true
   hostPort:
     enabled: true
     ports: {}
   nodeSelector:
     eks.amazonaws.com/nodegroup: <nginx-asg>

   service:
     enabled: false
     ports: {}
     nodePorts:
       http: <node-port>
     internal:
       ports: {}
       targetPorts: {}
    admissionWebhooks:
        nodeSelector:
          eks.amazonaws.com/nodegroup: <nginx-asg>
tcp:
  11701: <dih-namespace>/iidr-kafka:11701 #Smart DIH Digital Integration Hub.
An application architecture that decouples digital applications from the systems of record, and aggregates operational data into a low-latency data fabric. only
  3000:  <dih-namespace>/grafana:3000
  5432:  <dih-namespace>/xap-dgw-service:5432
  9092:  <dih-namespace>/kafka:9092  #Smart DIH only

Legend:

nginx-asg: A dedicated node group for an nginx instance.

dih-namespace: The namespace where Smart DIH is installed.

node-port: The host port, for example 30820.

Helm command used to install the Nginx controller calling the nginx-values.yaml:

helm install nginx nginx/ingress-nginx -n ingress --version=4.6.0 --values=nginx-values.yaml

Space High Availability (H/A) and Partitioning

The Space can be divided by two dimensions.

1. Partition - Adding more pods to extend the capacity of the Space.

2. Replication - Adding a backup pod per partition for h/a of the Space.

PU Parameter	Values	Description
partitions	1 or more	Divide the data among multiple pods.
h/a	true/false	Setting additional replication for each partition.
antiAffinity.enabled	true/false	When h/a is true, primary and backup pods will run in different hosts.
antiAffinity.topology	topology.kubernetes.io/zone	See note below.

Anti Affinity This describes the relationship between VMs and hosts or in Kubernetes between pods. Affinity will keep VM and hosts together and with Kubernetes, will keep pods on the same node. can operate in two ways:

Deploy primary and backup partitions on separate Kubernetes zones (default).

Deploy these two partitions on separate servers, but not necessarily on different zones.
For this second option, set.antiAffinity.topology must be set to kubernetes.io/hostname.

 

Tiered Storage

The following configuration must be completed to enable Tiered Storage:

1. Ensure that you have a suitable driver which will satisfy the required persistent volume, which depends on the cloud or file-system. More details can be found in the Kubernetes spec.

2. Enable the volume for Tiered Storage at the CRD level, using the parameters listed in the table below.

3. Build an image that includes a jar that is built according to the instructions found in this Tiered Storage page.

4. Follow the instructions for Deploying a Custom Processing Unit to load the jar.

The following parameters should be set in order to set the volume in the CRD level:

PU Parameter	Values	Description
persistence.enabled	true/false	Enable the persistence at the pod level
persistence.storageClassName	One of the cloud's storage classes	Storage Class Name (refer to Kubernetes documentation)
persistence.accessMode	ReadWriteOnce/ReadWriteMany	Storage Access mode (refer to Kubernetes documentation)
persistence.size	Example: "1Gi"	Depending on the expected amount of your total data (not cached data)
properties[0].name	Any key	Key of a property passed to the space jar (refer to number 3 above)
properties[0].value	Any value	Value of key[0]
properties[n].name	Key #n	For additional properties names
properties[n].value	Value #n	For additional properties corresponding values

Space Anti-Affinity

In h/a mode, the partition replications have to be placed in separate nodes or zones.

Parameter	Values	Description
operator.antiAffinity.topology	topology.kubernetes.io/zone or topology.kubernetes.io/host	The topology by which kubernetes activate the anti-affinity rule in the zone or the host

PU Parameter	Values	Description
antiAffinity.enabled	true/false	Enabling the anti-affinity for the specific PU

Manager Anti-Affinity

In a manager h/a scenario, 3 managers have to be set up for anti-affinity mode.

Parameter	Values	Description
manager.ha	true/false	Set up 3 managers (1 otherwise)
manager.affinity.enable	true/false	Apply anti-affinity rule upon scheduling

Environment and Properties that can be injected into a PU

Properties

There are two ways to set properties:

1. As an array by setting the properties value at the PU level.

2. Using valueFrom enable to point to a specific value in a config map (or secret):

properties:
  - name: SPECIAL_LEVEL_KEY
    valueFrom:
      configMapKeyRef:
        name: special-config
        key: special.how

3. Using propertiesFrom enable to reference a configmap or secret:

   propertiesFrom:
     - configMapRef:
         name: special-config

Environment Variables

Similar to properties, there are the same three options:

1. As an array of env variables (as in a container) using env parameter.

2. Using valueFrom enable to point to a specific value in a config map (or secret).

3. Using envFrom enable to reference a configmap of a secret.

StatefulSet Extensions

The GigaSpaces operator creates a StatefulSet out of values given in the PU manifest. These values are used for handling the installation and various other conditions (for example, scale), by the operator. The user might want to have additional settings on the StatefulSet. These settings are integrated as-is to the created StatefulSet.

Parameter	Values	Description
statefulSetExtension.enabled	true/false	Enable extension of the StatefulSet

The list of fields listed below are those that are allowed to be added. They can be added in the statefulSetExtension area in the chart/value file. Fields that are not listed are ignored by the operator.

StatefulSet Field Path	Description	Merge Policy*	Field-PK for Collision
spec.template.metadata.annotation	StatefulSet annotations	Merge with PU precedence	key
spec.template.metadata.finalizers	Pod controlled deletion	Merge with PU precedence	whole object
spec.template.metadata.labels	Labels of pods	Merge with PU precedence	key
spec.template.spec.containers	Containers running in pod	Merge with PU precedence	name
spec.template.spec.initContainers	Containers initializing	Merge with PU precedence	name
spec.template.spec.securityContext	Security pod definitions	Override	-
spec.template.spec.volumes	Volumes for containers	Merge with PU precedence	name

Probes

Liveness and Readiness probes are the means for the Kubernetes and GigaSpaces Manager to check the health of the PU.  For a Space, the operator sets the default probes if they are not provided in the settings of the PU manifest. These defaults ensure that the manager has a way to detect a failing partition of the Space.  If the user wishes to override this, for the case where there is additional logic that occurs at start-up of a PU, the probes can be set and support the URL. It is important to note that these endpoints still need to reflect the Space readiness.

PU Parameter	Values	Description
livenessProbe	Same as kubernetes spec	See Probes Spec. Description above.
readinessProbe	Same as kubernetes spec	See Probes Spec. Description above.

SpaceDeck Settings

Refer to the SpaceDeck pages.

Security Settings

The security server runs alongside the grid manager and is responsible for handling security related requests coming from inside or outside the cluster. The list below is for the security server cluster configuration.  Details of the end-user operation related to security can be found in the SpaceDeck Login and SpaceDeck Roles Management pages.

Parameter	Values	Description
manager.securityServer.enabled	true/false	Enable security at the backend
manager.securityServer.ingress.enabled	true/false	Adding an ingress to enable access from IDP An identity provider, or IDP, stores and manages users' digital identities. IDP and SSO can work together to authenticate users.
manager.securityServer.secretKeyRef.name	“root-credentials” or any secret name	Reference to a secret that holds the root credentials
manager.securityServer.rolesConfigMap	“roles-map” or other cm that has the roles	Reference to a configmap that holds the roles

Roles and Policies Configuration

A list of roles and their corresponding policies can be found in the "roles map" config map. In the example below, a user that has the role ROLE_ADMIN has specified policies that are defined for that role.  Different roles, for example ROLE_MNGR, would have different policies defined.

roles.ROLE_ADMIN: |
  SystemPrivilege MANAGE_IDENTITY_PROVIDERS
  , MonitorPrivilege MONITOR_PU
  , MonitorPrivilege MONITOR_JVM Java Virtual Machine. A virtual machine that enables a computer to run Java programs as well as programs written in other languages that are also compiled to Java bytecode.
  , GridPrivilege MANAGE_GRID
  , GridPrivilege MANAGE_PU
  , GridPrivilege PROVISION_PU
  , SpacePrivilege EXECUTE
  , SpacePrivilege ALTER
  , SpacePrivilege WRITE
  , SpacePrivilege READ
  , SpacePrivilege NOT_SET
  , SpacePrivilege TAKE
  , SpacePrivilege CREATE
  , SystemPrivilege MANAGE_ROLES