Deploying and Managing GigaSpaces in Kubernetes

This topic describes how to deploy GigaSpaces products in a Kubernetes environment. The integration is packaged as a Helm chart. You can deploy the full GigaSpaces platform, using the Helm chart available in the GigaSpaces Helm repository.

Prerequisites

The topics in this section assume basic knowledge of the GigaSpaces platform. If you aren't familiar with GigaSpaces, review the contents of the general Getting Started section before performing the tasks described here.

Before beginning to work with the Space and the GigaSpaces platform, ensure that you have the following installed on your local machine or a VM:

  • Helm

  • Kubernetes cluster (cloud, on-premise, or local via minikube)

This topic describes how to do basic and intermediate-level deployment tasks for the Space and the GigaSpaces platform, using simple Helm commands. You can perform these tasks using a Kubernetes minikube, because you only need a single node.

Deploying and Managing a Space

Accessing the GigaSpaces Helm Charts

The Helm package manager is used for installing the GigaSpaces platform in the Kubernetes environment.

A Helm chart can be used in a variety of formats and locations; packaged, unpackaged, accessed via a remote URL or even in a chart repository. The xap, xap-manager, xap-operator, xap-pu, insightedge, insightedge-manager, insightedge-operator, and insightedge-pu Helm charts are published in the GigaSpaces Helm charts repository at https://resources.gigaspaces.com/helm-charts.

Add a Helm Repo to the Repo List

You can point to the GigaSpaces Helm repo, so that Helm can locate the xap and insightedge charts for installation:

helm repo add gigaspaces https://resources.gigaspaces.com/helm-charts

If you're using minikube, note that the default kubernetes services created by GigaSpaces uses LoadBalancer, which requires running 'minikube tunnel' in the background. For more information see minikube tunnel.

Another option is to fetch the GigaSpaces Helm charts that you need and unpack them locally, so you don’t have to repeat the repo name and package version in each command (which has the added benefit of making the commands shorter). For example, you can pull (Helm 3) and unpack the Helm chart using the following command:

helm pull gigaspaces/xap-pu --version=16.1.1 --untar

The chart is unpacked in a local folder called insightedge, which is referenced in the Helm commands.

For general information of how to use Helm commands, refer to the Helm documentation.

helm <<command>> <<name-of-helm-chart>> <<parameters>>

You must fetch every chart that you will be using (for example xap, xap-pu and xap-manager) in your GigaSpaces application environment.

Deploying the GigaSpaces Platform

In order to prepare the environment for use, first the Platform Manager must be deployed. This deploys two pods: the Manager pod, and the Operator pod.

Note that Platform Manager is deployed by using a Helm chart, but further operations are performed using the UI, CLI or REST-API tools.

Type the following Helm command to create a Management Pod called testmanager that deploys Manager and Operator pods:

Method 1 (Preferred) – Umbrella Installation

After adding the GigaSpaces Helm repo, you can install the required chart(s) by referencing the chart name and product package version.

The example command format is shown below.

helm install umbrella gigaspaces/xap --version 16.1.1 --set metrics.enabled=false,manager.metrics.enabled=false,pu.metrics.enabled=false

Monitoring the Cluster in Kubernetes

You can monitor the cluster you deployed using any of the following administration tools.

  • Helm: Run the following command to print the status of the "demo' release in the command window.
helm status umbrella
  • Kubernetes dashboard: run the following command to open a dashboard in your browser, where you can see all the Pods and services by clicking the various tabs. For example, if you're using minikube:
minikube dashboard
  • Kubectl: run the following command to print the name, description, and status of the Pods in the command window. A list of events is also printed, which can be used for troubleshooting purposes. For example, if you detected a problem in one of the Pods, you can see the Pod termination reason and exit code.
kubectl describe pod

Additional Helm Charts Available in the GigaSpaces Repo

Helm PU/Space deploy:



helm install demo gigaspaces/xap-pu --version 16.1.1 –-set manager.name=umbrella,ha=true,partitions=1,antiAffinity.enabled=true,properties="pu.dynamic-partitioning=true"

Helm PU/Scale out/in:


kubectl patch pu demo -p ‘{“spec”:{“partitions”: 2}}’ --type=merge

Method 2 (Alternative Installation) – Step-By-Step

Before using these charts for the first time, you must fetch the chart, as described in Deploying and Managing GigaSpaces in Kubernetes above.

  • Use a basic Helm chart to start Manager, and then use GigaSpaces Tools (Ops Manager/CLI/API) for Processing Units management. This method uses Kubernetes Operator and provides all GigaSpaces functionality. It is the default and is described below.

Ensure that no CRDs are installed. Test this with kubectl get pus.
If there are PUs installed, and they are not in the Deployed state, Operator will continue trying to complete the installation until the processing unit is installed.

Install Manager

helm install manager gigaspaces/xap-manager --version 16.1.1 --set metrics.enabled=false

Install Operator, Webhook and CRD Definition

helm install operator gigaspaces/xap-operator --version 16.1.1 --set manager.name=manager

Install PU

helm install pu gigaspaces/xap-pu --version 16.1.1 --set manager.name=manager --set metrics.enabled=false

Ensure That All Pods are Running

Run kubectl get pus and ensure that the deployed processing unit is in the Deployed state.

 

Customizing the Helm Chart

The following are examples of providing custom capabilities in the YAML file.

Define a Statefulset:


statefulSet:
    metadata:
      name: {{ .Release.Name }}
      namespace: {{ .Release.Namespace }}
      labels:
        app.kubernetes.io/managed-by: gigaspaces-operator
    spec:
      selector:
        matchLabels:
          selectorId: {{ .Release.Name }}
      template:
        spec:
          {{- if  .Values.statefulSet.serviceAccountName }}
          serviceAccountName: "{{ .Values.statefulSet.serviceAccountName}}"
          {{- end }}
          {{- if  .Values.statefulSet.restartPolicy }}
          restartPolicy: "{{  .Values.statefulSet.restartPolicy }}"
          {{- else }}
          restartPolicy: "Always"
          {{- end }}
          {{- if  .Values.statefulSet.terminationGracePeriodSeconds }}
          terminationGracePeriodSeconds: {{  .Values.statefulSet.terminationGracePeriodSeconds }}
          {{- else }}
          terminationGracePeriodSeconds: 30
          {{- end }}
          ## new part
          {{- if  .Values.statefulSet.initContainers.enabled }}
          initContainers:
            - name: aws-cli
              image: "{{  .Values.statefulSet.initContainers.image.repository }}:{{  .Values.statefulSet.initContainers.image.tag }}"
              command: {{  .Values.statefulSet.initContainers.command }}
              volumeMounts:
                - mountPath: /artifacts
                  name: artifacts
                  readOnly: false
            - name: check-manager-ready
              image: "{{  .Values.image.repository }}:{{  .Values.image.tag }}"
              command: ["tools/kubernetes/entrypoint.sh"]
              args:
                - "component=init"
                - "verbose=true"
                - "name={{  .Release.Name }}"
                {{ if ( .Values.license) }}- "license={{  .Values.license}}"{{- end }}
                {{ if ( .Values.manager.name) }}- "manager.name={{ .Values.manager.name}}"{{- end }}
                {{ if ( .Values.manager.ports.api) }}- "manager.ports.api={{ .Values.manager.ports.api}}"{{- end }}
                {{ if ( .Values.manager.discoveryTimeoutSeconds) }}- "manager.discoveryTimeoutSeconds={{ .Values.manager.discoveryTimeoutSeconds}}"{{- end }}
            - name: check-external-service-ready
              image: "{{  .Values.image.repository }}:{{  .Values.image.tag }}"
              command: ["tools/kubernetes/entrypoint.sh"]
              args:
                - "component=init-pu-external-service"
                - "verbose=true"
                - "release.namespace={{  .Release.Namespace }}"
          {{- end }}
          ## new end
          securityContext:
            {{- if  .Values.statefulSet.securityContextFsGroup }}
            fsGroup: {{  .Values.statefulSet.securityContextFsGroup }}
            {{- else }}
            fsGroup: 2000
            {{- end }}
  {{- end }}

Configuration Options

The following configuration options can be specified when using a chart:

  • ha=true/false — each primary Data Pod has one backup Data Pod. Default is false.

  • metrics-enabled=true/false — enable or disable Grafana metrics. Default is true.

  • antiAffinity.enabled=true/false — spread the partitions across nodes. Default is false.

Troubleshooting

Problem:

  • Cannot delete pus (kubectl delete pus <pu name>), the delete command gets hung. This may occur when attempting to delete a processing unit and the CRD causes a system hang.

Resolution:

kubectl patch pus/pu -p '{"metadata":{"finalizers":[]}}' --type=merge

See here for more information.

Problem:

  • Pod is not initializing but kubectl get pus shows it is in deployed state.

Resolution:

Execute kubectl logs <operator pod>. The logs will show custom and merged values. Check for problems with the merged values.