Multi-Region Replication in Kubernetes

For implementation examples, see https://github.com/Gigaspaces/solution-hub/tree/master/wan-gateway-kubernetes.

 

You can implement multi-region replication in Kubernetes by deploying a GigaSpaces WAN gateway. The Kubernetes-based WAN gateway has all the functionality of the service-grid based WAN gateway, and provides multi-region replication for fast and efficient disaster recovery.

This topic describes how to set up a WAN gateway for a GigaSpaces environment in cloud-based Kubernetes clusters. You have to create two clusters, one for each region that will host GigaSpaces, deploy GigaSpaces on each cluster, and then deploy a WAN gateway for each region.

Setting Up the WAN Gateway

 

When defining a GigaSpaces WAN gateway for deployment in Kubernetes, you should always set it up in the following order:

  1. Manager for each geographical zone
  2. Space for each geographical zone
  3. WAN gateway for each geographical zone, allowing remote lrmi

The procedure described in this topic creates a WAN gateway environment for two clusters, one located geographically in the Germany ("central"), and the other in USA ("west"). You will need the GigaSpaces Helm charts, and pu.xml files for the delegator, sink, and WAN gateway components.

Initializing Helm

Helm must be installed before you download the GigaSpaces Helm chart.

To deploy the Helm Tiller service, type the following command:

helm init --service-account tiller

Installing the GigaSpaces Manager per Geographical Site

The Manager is always installed first, followed by the service(s) that it will manage.

To install the two Managers:

  1. You can use the default configuration to install the Manager for each geographical location. Use the following command to install a Manager for the central location:

    helm install gigaspaces/xap-manager --name mymanager --version=15.2.0--namespace gigaspaces-central-ns
    --set image.repository=?????,image.tag=?????15.0

     

  2. Repeat step 2 for the second geographical location, west, using namespace of gigaspaces-west-ns.

if you create the cluster, the cluster config is automatically added to your ~/.kube/config.

Select the context to switch between environments, using the commands kubectl config get-contexts and kubectl config use-context.

 

To check the status of the host and services (to ensure that the Managers have been installed properly), use the following commands:

kubectl get services -n gigaspaces-central-ns

kubectl get services -n gigaspaces-west-ns

An example of kubectl output is shown below.

Note that the central manager host is external ip address is identified as:

a8e09518a36e111eabe4f06ed48642f1-815235200.ca-central-1.elb.amazonaws.com

and the west manager host external ip address is identified as:

abb7ac8f136e211ea8b1d06a0d35ff23-880727685.eu-west-2.elb.amazonaws.com

Installing the GigaSpaces Space Service

The next step is to install a Space service (Processing Unit) for each geographical site, which is defined in the service JAR as described in Multi-Region Replication for Disaster Recovery. It is important to verify that all of the WAN gateway-related properties have been configured correctly for each geographical location.

Before deploying the space pu.jar, make sure the pu.properties is properly configured.

For example, for the central Space:

localSpaceName=mySpace
local-gateway-name=central
remote-gateway-name=west

And for the Germany Space:

localSpaceName=mySpace
local-gateway-name=west
remote-gateway-name=central

For more information about defining the service JAR, refer to Configuring a Space With Gateway Targets.

 

Install the central space

helm install gigaspaces/xap-pu --name myspace-ca --version=15.0.0 --set manager.name=mymanager,partitions=1,resourceUrl=https://github.com/dixsonhuie/mirrorexample2/blob/master/iks/wan-space-central.jar?raw=true --set service.lrmi.enabled=true --set image.repository=dixsonhuie/xap-enterprise-wan-logging,image.tag=15.0 --namespace gigaspaces-central-ns

 

Install the west space

helm install gigaspaces/xap-pu --name myspace-gb --version=15.0.0 --set manager.name=mymanager,partitions=1,resourceUrl=https://github.com/dixsonhuie/mirrorexample2/blob/master/iks/wan-space-west.jar?raw=true --set service.lrmi.enabled=true --set image.repository=dixsonhuie/xap-enterprise-wan-logging,image.tag=15.0 --namespace gigaspaces-west-ns

 

Before deploying the gateway pu.jar, make sure the pu.properties is properly configured

Example for pu.properties central

local-gateway-name=central

local-spaceUrl=jini://*/*/mySpace

local-lookup-host=a8e09518a36e111eabe4f06ed48642f1-815235200.ca-central-1.elb.amazonaws.com

local-lookup-port=4174

local-communication-port=8200

remote-gateway-name=west

remote-lookup-host=abb7ac8f136e211ea8b1d06a0d35ff23-880727685.eu-west-2.elb.amazonaws.com

remote-lookup-port=4174

remote-communication-port=8200

 

Example for pu.properties west

local-gateway-name=west

local-spaceUrl=jini://*/*/mySpace

local-lookup-host=abb7ac8f136e211ea8b1d06a0d35ff23-880727685.eu-west-2.elb.amazonaws.com

local-lookup-port=4174

local-communication-port=8200

remote-gateway-name=central

remote-lookup-host=a8e09518a36e111eabe4f06ed48642f1-815235200.ca-central-1.elb.amazonaws.com

remote-lookup-port=4174

remote-communication-port=8200

 

These properties will be used to override the property placeholders under the gatewayLookups section. This host and discovery-port are the host and are exposed by Kubernetes in the helm gateway install.

<os-gateway:lookups id="gatewayLookups">

<os-gateway:lookup gateway-name="${local-gateway-name}" host="${local-lookup-host}" discovery-port="${local-lookup-port}" communication-port="${local-communication-port}"/>

<os-gateway:lookup gateway-name="${remote-gateway-name}" host="${remote-lookup-host}" discovery-port="${remote-lookup-port}" communication-port="${remote-communication-port}"/>

</os-gateway:lookups>

 

The host and discovery port are the host and ports exposed by Kubernetes in the Helm manager install.

Install the gateway

helm install gigaspaces/xap-pu --name gateway-ca --version=15.0.0 --set manager.name=mymanager,resourceUrl=https://github.com/dixsonhuie/mirrorexample2/blob/master/iks/wan-gateway-central.jar?raw=true --set service.lrmi.enabled=true --set java.options="-Dcom.sun.jini.reggie.initialUnicastDiscoveryPort=4174\,-Dcom.gs.transport_protocol.lrmi.bind-port=8200" --set image.repository=dixsonhuie/xap-enterprise-wan-logging,image.tag=15.0 --namespace gigaspaces-central-ns

 

helm install gigaspaces/xap-pu --name gateway-gb --version=15.0.0 --set manager.name=mymanager,resourceUrl=https://github.com/dixsonhuie/mirrorexample2/blob/master/iks/wan-gateway-west.jar?raw=true --set service.lrmi.enabled=true --set java.options="-Dcom.sun.jini.reggie.initialUnicastDiscoveryPort=4174\,-Dcom.gs.transport_protocol.lrmi.bind-port=8200" --set image.repository=dixsonhuie/xap-enterprise-wan-logging,image.tag=15.0 --namespace gigaspaces-west-ns

 

kubectl exec -it gateway-gb-xap-pu-0 -n gigaspaces-west-ns --bash

If you have to take down a Space and then redeploy it, the public IP address will change. The WAN gateway service JAR must be updated accordingly.

Enable LRMI as follows:

-service.lrmi.enabled=true

Testing and Monitoring the WAN Gateway

To test whether your data is replicating between your clusters, you can use a Java client or Apache Zeppelin to write data to the source and then query the target.

To see information about your WAN gateways, you can view the system logs.