XAP Skyline

Smart DIH

Tiered Storage Configuration for Kubernetes

Preparing a Kubernetes Cluster

These instructions also apply for OpenShift.

For a Tiered StorageClosed Automatically assigns data to different categories of storage types based on considerations of cost, performance, availability, and recovery. overview refer to Intelligent Tiering Overview.

  1. Make sure worker nodes support the required storage type.

  2. Create a new storage class zrs_sc.yaml file, for example:

     kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
	name: azure-premium-zrs
provisioner: disk.csi.azure.com # replace with "kubernetes.io/azure-file" if AKS version is less than 1.21
allowVolumeExpansion: true
parameters:
	skuName: Premium_ZRS
  3. Deploy this resource:
    4.  kubectl apply -f zrs_sc.yaml

  4. Once it is deployed, make sure the new storage class is available and it is named azure-premium-zrs.

    5.  $ kubectl get sc
NAME                    	     PROVISIONER          	RECLAIMPOLICY   	VOLUMEBINDINGMODE      ALLOWVOLUME   EXPANSION   AGE
azure-premium-zrs          file.csi.azure.com   	Delete          		      Immediate              			      true                   		4d12h
azurefile-csi           	     file.csi.azure.com   	 Delete          			Immediate              				true                   		 11d
managed-csi (default)    disk.csi.azure.com   	   Delete          			  WaitForFirstConsumer   	     true                   	     11d

  5. Deploy the space with tiered storage enabled, and specify the required storage class name ('azure-premium-zrs').

Deploying a Space with Tiered Storage

When deploying with Tiered Storage, SpaceClosed Where GigaSpaces data is stored. It is the logical cache that holds data objects in memory and might also hold them in layered in tiering. Data is hosted from multiple SoRs, consolidated as a unified data model. needs to be marked to be tiered storage and for every type when it is registered, if you want to cache some data you have to specify a cache rule. In the example below, space-config.engine.cache_policy = 4 is for Tiered Storage.

Run the following KubernetesClosed An open-source container orchestration system for automating software deployment, scaling, and management of containerized applications. Helm command:

helm install ${SPACE_NAME} ${REPO}/xap-pu --version ${GS_VERSION} --set properties\[0\].name=\"space-config.engine.cache_policy\",properties\[0\].value=\"4\",persistence.enabled=true, persistence.storageClassName=${STORAGE_CLASS},persistence.accessMode=ReadWriteOnce,persistence.size=${STORAGE_SIZE},env\[0\].name=GS_SA_PRAGMA_SYNCHRONOUS,env\[0\].value=\"${SQLITE_PRAGMA_SYNCHRONOUS}\",resources.limits.memory=${MEMORY_SIZE},partitions=${PARTITIONS},ha=${HA}

Parameters

Parameter Description
STORAGE_CLASS Corresponds to the storage class name available for on disk data, for example ‘azure-premium-zrs
SQLITE_PRAGMA_SYNCHRONOUS Recommended value is 1 (NORMAL). All supported pragma statements: https://www.sqlite.org/pragma.html, search for ‘PRAGMA schema.synchronous’. For any changes to the predefined configuration, please contact our technical support team.
STORAGE_SIZE The storage size allocated for Warm (on-disk) data per partition. This will internally create a PersistentVolumeClaim resource with the specified size for each space partition. The cluster must have this amount of storage available for the corresponding storage class, e.g. 10Gi
MEMORY_SIZE Space partition memory size, e.g. 12G
PARTITIONS The number of space partitions (excluding backup partitions), e.g. 4
HA High Availability true/false, when set to true, append also ,antiAffinity.enabled=true

Tiered Storage Configuration for Space Types

Registering a Space Type with Tiered Storage Configuration

In the Java example below, take note of the highlighted section.

SpaceTypeDescriptor typeDescriptorDoc1 = 
   new SpaceTypeDescriptorBuilder("SpaceTypeExample")
	.addFixedProperty("id", Integer.class)
	.addFixedProperty("field1", String.class)
	.addFixedProperty("field2", String.class)
	.addFixedProperty("field3", String.class)
	.addFixedProperty("expireDate", LocalDateTime.class)
	.idProperty("id")
	.supportsDynamicProperties(false)
	.setTieredStorageTableConfig(
		new TieredStorageTableConfig()
			>.setName("SpaceTypeExample")// set to space type name
			>.setCriteria("field1='1'")// see examples below
	)
	.create()

Tiered Storage Policies in Type Registration

Example Hot/Warm Tiers Distribution
new TieredStorageTableConfig() .setName("SpaceTypeExample") .setCriteria("field1='1'") Note: Supported operators: = != >= <= and or

Criteria-Based

Hot TierClosed Part of GigaSpaces Tiered Storage Mechanism. Hot Tier is RAM and is used for priority data and read/write operations that require the fasted access Matching provided criteria
Warm TierClosed Part of GigaSpaces Tiered Storage Mechanism. Warm tier is SSD (recommended media) storage which is mostly used for data that is read-only and is less frequently used. All
new TieredStorageTableConfig() .setName("SpaceTypeExample") .setTimeColumn("expireDate") .setPeriod(Duration.ofMinutes(60))

Time-Based

Hot Tier Matching condition expireDate+60min<now
Warm Tier All

Note: Only the initial value of the expireDate column will define Hot Tier retention of an object.
new TieredStorageTableConfig() .setName("SpaceTypeExample") .setTransient(true)

Memory Only

Hot Tier All
Warm Tier None
null (*) or no call to: .setTieredStorageTableConfi

Warm Tier Only

Hot Tier None
Warm Tier All
new TieredStorageTableConfig() .setName("SpaceTypeExample") .setCriteria("all")

Hot and Warm

Hot Tier All
Warm Tier All

Limitations and Recommendations

Supported Not Supported

  • Broadcast Table: Recommended all (both disk and cache).

  • At this stage, primitive property types and selected Java classes are supported in Tiered Storage.

  • Only native value types (integer, string, double, byte, date types, array, etc.).

  • SpaceDocument: Only fixed properties.

  • The number of backups per partition is zero or one.

  • Transaction Support: Local transactions are supported Local View.

  • Kubernetes.

  • Dynamic Indexes.