Interacting with the Space


In this part of the tutorial we will demonstrate how to create a Space and how you can interact with it. We will also demonstrate how you can improve your Space search performance by using indexes and returning partial results.

Creating a Space

Let’s create a Space called ‘xapTutorialSpace’ that is co-located within an application. This type of Space is called embedded Space.

Here is an example how you start an embedded Space:

using GigaSpaces.Core;

// Create the SpaceProxy
ISpaceProxy spaceProxy = new EmbeddedSpaceFactory("xapTutorialSpace").Create();

This Space we just created can also be accessed remotely from another application. In order to do so you would use the following code:

ISpaceProxy spaceProxy = new SpaceProxyFactory("xapTutorialSpace").Create();

You can configure the Space URL with several options.

When a client connects to a Space, a proxy is created that holds a connection which implements the Space API. All client interaction is performed through this proxy.

With XAP you can also create a Local Cache and a Local View.

  • Local Cache : This client side cache maintains any object used by the application. The cache data is loaded on demand (lazily), based on the client application’s read operations.
  • Local View : This client side cache maintains a specific subset of the data. The subset is predefined by the user. The cache is populated when the client application is started. In both cases, updates are performed (objects are added/updated/removed) on the master Space, the master Space then propagates the changes to all relevant local views and caches.

These two scenarios are only applicable for remote clients.

The Space Object

XAP supports two types of objects that can interact with the Space, PONOs and Documents.

PONO

Any PONO can be used to interact with the Space. The PONO needs to implement a default constructor, setters and getters for every property you want to store in the Space.

using System;
using System.Collections.Generic;

using GigaSpaces.Core.Metadata;

	public class User {

		[SpaceID(AutoGenerate = false)]
		[SpaceRouting]
		public long? Id { set; get; }
		public String Name{ set; get; }
		public double? Balance{ set; get; }
		public double? CreditLimit{ set; get; }
		public Nullable<EAccountStatus> Status{ set; get; }
		public Address Address{ set; get; }
		public String[] Comment{ set; get; }
		public Dictionary<String, String> Contacts{ set; get; }
		public List<Group> Groups{ set; get; }
		public List<int?> Ratings{ set; get; }

	    //.....
	}
}

The SpaceId

The Space generates a unique identifier (UID) for every object in one of the following ways:

  • When a Space object has no SpaceId attribute declared, the Space auto-generates a UID for the object.
  • When a Space object has an attribute which is declared as SpaceId and marked as auto-generate=false, the UID is generated based on the value of the ID attribute the user is setting.
  • When a Space object has an attribute which is declared as SpaceId and marked as auto-generate=true, the UID is generated by the Space and placed back into the attribute using the relevant setter method. In this case, the attribute must be of String type.
See also:

Space Object Id

Defining Routing

Partitioning is used when the total number of objects is too big to be stored in a single Space. In this case we will divide the data into several partitions. By designating an attribute on the Space class as a partitioning key, the Space proxy will then know to which partition a particular instance of the Space class belongs to. The Space proxy uses the entry’s routing attribute hash code to determine the corresponding partition for it.

The routing attribute can be explicitly set using the [SpaceRouting] annotation for PONO entries or via the SpaceTypeDescriptorBuilder for document entries. If the routing attribute is not explicitly set, the Space id attribute is used for routing. If the Space id attribute is not defined, the first indexed attribute (alphabetically) is used for routing, otherwise the first attribute (alphabetically) is used for routing.
See also:

Data Partitioning

Space Document

The XAP document API exposes the Space as Document Store. A document, which is represented by the class SpaceDocument, is essentially a collection of key-value pairs, where the keys are strings and the values are primitives, String, Date, other documents, or collections thereof. Unlike PONOs, which force users to design a fixed data schema (in the form of a class definition) and adhere to it, a document is much more dynamic, users can add and remove properties at runtime as necessary. A Document always belongs to a certain type, represented by the class SpaceTypeDescriptor.

To create a document we use a Dictionary for its properties. The SpaceDocument object is instantiated by using the type name and properties. XAP provides a special implementation of a Dictionary called DocumentProperties.

Here is an example how you can create a SpaceDocument:

using System;

using GigaSpaces.Core;
using GigaSpaces.Core.Document;
using GigaSpaces.Core.Metadata;

namespace xaptutorial.model
{
	public SpaceDocument createDocumemt() {
		DocumentProperties properties = new DocumentProperties ();

		properties["CatalogNumber"]= "av-9876";
		properties["Category"]= "Aviation";
		properties["Name"] = "Jet Propelled Pogo Stick";
		properties["Price"]= 19.99;
		properties["Tags"]= new String[4] {"New", "Cool", "Pogo", "Jet"};

		DocumentProperties p2 = new DocumentProperties();
		p2["Manufacturer"]="Acme";
		p2["RequiresAssembly"]=true;
		p2["NumberOfParts"]= 42;
		properties["Features"]=p2;

		SpaceDocument document = new SpaceDocument("Product", properties);
		proxy.Write(document);

		return document;
	}
}

In order to use the SpaceDocument, we need to register its schema first with the Space:

public void registerProductType() {
	// Create type Document descriptor:
	SpaceTypeDescriptorBuilder typeBuilder = new SpaceTypeDescriptorBuilder("Product");
	typeBuilder.SetIdProperty("CatalogNumber");
	typeBuilder.SetRoutingProperty("Category");
	typeBuilder.AddPathIndex("Name");
	typeBuilder.AddPathIndex("Price", SpaceIndexType.Extended);
	ISpaceTypeDescriptor typeDescriptor = typeBuilder.Create();

	// Register type:
	proxy.TypeManager.RegisterTypeDescriptor(typeDescriptor);
}

Only properties with special roles like ID and Routing are part of the schema definition. These meta model settings cannot be changed without restarting the Space or dropping the type, clearing all its instances and reintroducing it again.

It is possible to write a PONO to the Space and read it back as a document, and vice versa. This scenario is useful when you want to read or modify PONO objects without loading the concrete C# classes.

Interacting with the Space

All Space operations are relevant to both the PONO and Document.

Writing an object to Space:

When writing an object to the Space, the object is created in Space if it does not exist. If it already exists in Space it will be updated. This is the default behavior of the write operation.

public void writeUser() {
     User user = new User();
     user.Id= 1L;
     user.Name="John Smith";
     user.Status=EAccountStatus.ACTIVE;

     // Write the user to the Space
     spaceProxy.Write(user);
}

It is also possible to write multiple objects in one operation to the Space (batch mode). This can vastly improve the performance if you need to load many objects at once into the Space.

Here is an example on how you write multiple objects to the Space:

public void writeUsers() {
     User[] users = new User[2];
     users[0] = new User();
     users[0].Id=1L;
     users[0].Name="John Dow";
     users[0].Status=EAccountStatus.ACTIVE;

     users[1] = new User();
     users[1].Id=2L;
     users[1].Name="John Dow";
     users[1].Status=EAccountStatus.ACTIVE;

     spaceProxy.WriteMultiple(users);
}

There are several options to override the default behavior of the write operation. You can change the lifetime of an object by supplying a LEASE to the operation. You can also change the modifier on the operation to change the behavior.

Here is an example:

public void writeOnlyWithLease() {
	User user = new User();
	user.Id=1L;
	user.Name="John Smith";
	user.Status=EAccountStatus.ACTIVE;

    // no transaction and 10 seconds lease time
	proxy.Write(user,null,0,10000, WriteModifiers.WriteOnly);
}

In this example, we are writing an object to the Space with zero delay, 10 seconds to live and write only if the object does not already exist in the Space. If the object already exists, an exception will be thrown.

Updating an object in Space

When you want to update only a couple of attributes on an object in Space, you can use the change operation and update specific fields or even nested fields or modify collections and maps without having to supply the entire collection or map for the operation. With the following change operation example it is not necessary to read the object first from the Space to update it. The Change API reduces a normal two step operation to a one step operation. This operation can vastly improve performance when you have an object with many attributes and you only need to update one or a couple of attributes.

public void changeSet() {
	User user = new User();
	user.Id=1L;
	user.Name="John Dow";
	user.Status=EAccountStatus.ACTIVE;
	proxy.Write(user);

	IdQuery<User> idQuery = new IdQuery<User>(1L);
	IChangeResult<User> changeResult =
		proxy.Change<User>(idQuery,
		new ChangeSet().Set("Status", EAccountStatus.BLOCKED));

	if (changeResult.NumberOfChangedEntries == 0) {
		Console.WriteLine("Entry does not exist");
	}
}

There are several other change operations available; ‘increment’, ‘decrement’, ‘addToCollection’, ‘removeFromCollection’ etc.

See also:

The Change API

Querying the Space

Now we are ready to query the Space. XAP provides several ways to perform queries against the Space:

  • Query by ID
  • Query by Template
  • Query by SQL

Query by ID

This is the simplest and fasted way to retrieve objects from the Space.

Here is an example of a query by id:

public User findUserById() {
	return proxy.ReadById<User>(1L);
}

You can also perform a bulk read for multiple Id’s

public User[] findUsersByIds() {
	object[] ids  = new object[3]{ 1L, 2L, 3L };

	IReadByIdsResult<User> result = proxy.ReadByIds<User>(ids);
	return result.ResultsArray;
}

Query by Template

Template matching (match by example) is a simple way to query the Space. The template is a PONO of the desired entry type, and the attributes which are set on the template (i.e. not null) are matched against the respective attributes of entries of the same type in the Space. Attributes with null values are ignored (not matched).

The following examples assume the default constructor of the User class initializes all its attributes to null.

Read an entry of type User where the name is ‘John Dow’:

public User findUserByTemplate() {
	User user = new User();
	user.Name="John Dow";
	return proxy.Read(user);
}

You can also perform a bulk read with templates. In the example below will read all users that have a status of ACTIVE:

public User[] findUsersByTemplate() {
	User user = new User();
	user.Status=EAccountStatus.ACTIVE;
	return proxy.ReadMultiple(user);
}

Template Matching support inheritance relationships, so that entries of a sub-class are visible in the context of the super class, but not the other way around.

SQL Query

The SQLQuery class is used to query the Space with an SQL-like syntax. The query statement includes only the WHERE statement part. An SQLQuery is composed from the class of entry to query and an expression in SQL syntax.

public User[] sqlFindUsersByName() {
	SqlQuery<User> query = new SqlQuery<User>("Name = 'John Dow'");
	return proxy.ReadMultiple<User>(query);
}

public User[] sqlFindUsersByNameAndCreditLimit() {
	SqlQuery<User> query = new SqlQuery<User>("Name = 'John Dow' AND CreditLimit > 1000");
	return proxy.ReadMultiple<User>(query);
}

public User[] sqlFindUsersByNameAndIds() {
	SqlQuery<User> query = new SqlQuery<User>( "Name = 'John Dow' AND Id IN(1L,3L,5L)");
	return proxy.ReadMultiple<User>(query);

public User[] sqlFindUsersByNameLike() {
    SqlQuery<User> query = new SqlQuery<User>("Name like 'A%'");
    return proxy.ReadMultiple<User>(query);

}

Parameterized Queries

You can separate the values for the SQL criteria expression by placing a ‘?’ symbol instead of the actual value in the expression. When executing the query, the conditions that includes ‘?’ are replaced with the corresponding parameter values supplied via the setParameter method.

For example:

public User[] sqlParameterFindUsersByName() {
	SqlQuery<User> query = new SqlQuery<User>("Name = ?")
	query.SetParameter(1, "John Dow");
	return proxy.ReadMultiple<User>(query);
}

public User[] sqlParameterFindUsersByNameAndCreditLimit() {
	SqlQuery<User> query = new SqlQuery<User>("Name = ? AND CreditLimit > ?");
	query.SetParameter(1, "John Dow");
	query.SetParameter(2, 1000);
	return proxy.ReadMultiple<User>(query);
}

Nested property queries

Many times a class has embedded classes as attributes. You can query for attributes within the embedded classes. Matching a nested attribute is done by specifying a Path which describes how to obtain its value. For example, our user class has an embedded attribute of an Address that has a zipCode attribute.

Nested Objects

By default, nested objects are kept in a binary form inside the Space. In order to support nested matching, the relevant property should be stored as document, or as object if it is in an interoperability scenario and it has a corresponding Java class.

Here is an example how you would annotate a class to enable nested queries:

	public class User {

		[SpaceID(AutoGenerate = false)]
		[SpaceRouting]
		public long? Id { set; get; }

		public String Name{ set; get; }
		public double? Balance{ set; get; }
		public double? CreditLimit{ set; get; }
		public Nullable<EAccountStatus> Status{ set; get; }

        [SpaceProperty(StorageType = StorageType.Document)]
		public Address Address{ set; get; }
		public String[] Comment{ set; get; }

        [SpaceProperty(StorageType = StorageType.Document)]
		public Dictionary<String, String> Contacts{ set; get; }

        [SpaceProperty(StorageType = StorageType.Document)]
		public List<Group> Groups{ set; get; }

        [SpaceProperty(StorageType = StorageType.Document)]
		public List<int?> Ratings{ set; get; }

		//......
	}
}

Here is an example how you can query the Space for all users that have a zip code of ‘12345’.

public User[] sqlFindUsersByZipCode() {
	SqlQuery<User> query = new SqlQuery<User>("Address.ZipCode = 12345");
	return proxy.ReadMultiple<User>(query);
}

Nested Collections

It is possible to query embedded collections. Our user class has a collection groups that he belongs to. We can query the Space for all users that belong to a certain group:

public User[] findUsersByGroup() {
	SqlQuery<User> sqlQuery = new SqlQuery<User>( "Groups[*].Id = 1");
	return proxy.ReadMultiple<User>(sqlQuery);
}

There are several additional query options available. For example you can query Nested Maps by key,query with Regular Expression, Enum attributes and others.

See also:

SQL Query

Query returning partial results

In some cases when querying the Space for objects only specific attributes of an objects are required and not the entire object (delta read). For that purpose the Projection API can be used where you can specify which attributes are of interest and the Space will only populate these attributes with the actual data when the result is returned back to the user. This approach reduces network overhead and can vastly improve performance.

In this example below we are just interested in the name attribute of the user object:

public User[] findUsersByNameAndProjection() {
	SqlQuery<User> query = new SqlQuery<User>( "Name = ?"){Projections = new []{"Name"}};
	query.SetParameter (1, "John Dow");

	return proxy.ReadMultiple<User>(query);
}

Document Queries

You can also query the Space for documents. Just like the PONO queries, you can use query by ID, template and SQLQuery.

Here are some examples how you can query the Space for documents:

public SpaceDocument readProductById() {
	SpaceDocument template = new SpaceDocument("Product");
	template["CatalogNumber"]= "av-9876";
	return proxy.Read(template);
}

public SpaceDocument readProductByTemplate() {
	SpaceDocument template = new SpaceDocument("Product");
	template["Name"]= "Jet Propelled Pogo Stick";
	return proxy.Read(template);
}

public SpaceDocument[] readProductsBySQL() {
	SqlQuery<SpaceDocument> query = new SqlQuery<SpaceDocument>("Product","Price > ?");
	query.SetParameter(1, 15.0);
	return proxy.ReadMultiple<SpaceDocument>(query);
}
See also:

The Document API

LINQ Queries

XAP includes a custom LINQ provider, which enables developers to take advantage of their existing C# skills to query the Space without learning a new language. Here is an example :

using GigaSpaces.Core.Linq;

   var query = from p in spaceProxy.Query<Account>()
            where p.number == "12345"
            select p;

   foreach (var account in query)
   {
    // ...
   }
See also:

LINQ Query

Removing Objects from Space

To remove objects from a Space you can use the take or the clear operation.

Take operation

The take operation returns an object and removes it from the Space. XAP provides several options for the take operation:

  • Take by ID
  • Take by template
  • Take by SQLQuery
  • Take multiple

Here are some examples:

public User takeUserById() {
   return spaceProxy.TakeById<User>(1L);
}

public User takeUserByTemplate() {
   User template = new User();
   template.Name="John Dow";
   return spaceProxy.Take<User>(template);
}

public User[] takeUsersBySQL() {
     SqlQuery<User> query = new SqlQuery<User>("Status = ?");
     query.setParameter(1, EAccountStatus.BLOCKED);
   return spaceProxy.TakeMultiple<User>(query);
}

Clear operation

The clear operation removes objects from a Space without returning them. Here are some examples:

public void clearUserByTemplate() {
     User template = new User();
     spaceProxy.Clear(template);
}

public void clearUserBySQL() {
     SqlQuery<User> query = new SqlQuery<User>(User.class, "Name = ?");
     query.setParameter(1, "John Dow");
     spaceProxy.Clear(query);
}

// clear all objects in space
public void clearAllObjectInSpace() {
     spaceProxy.Clear(null);
}

Aggregation

The Aggregators allow you to perform the entire aggregation activity at the Space side avoiding any data retrieval back to the client side. Only the result of each aggregation activity performed with each partition is returned back to the client side where all the results are reduced and returned to the client application.

using GigaSpaces.Core.Linq;

...
var queryable = from p in spaceProxy.Query<Person>("Country='UK' OR Country='U.S.A'") select p;
// retrieve the maximum value stored in the field "age"
int maxAgeInSpace = queryable.Max(p => p.Age);
// retrieve the minimum value stored in the field "age"
int minAgeInSpace = queryable.Min(p => p.Age);
// Sum the "age" field on all space objects.
int combinedAgeInSpace = queryable.Sum(p => p.Age);
// Sum's the "age" field on all space objects then divides by the number of space objects.
double averageAge = queryable.Average(p => p.Age);
// Retrieve the space object with the highest value for the field "age".
Person oldestPersonInSpace = queryable.MaxEntry(p => p.Age);
// Retrieve the space object with the lowest value for the field "age".
Person youngestPersonInSpace = queryable.MinEntry(p => p.Age);

XAP also supports, Compound and Embedded Fields Aggregation.

See also:

Aggregators

Indexing

To improve performance, it is possible to index one or more attributes for an object. The Space maintains additional data for indexed attributes, which shortens the time required to determine a match resulting in improved performance. However, indexes consume more resources and impacts the write operations performance.

Inheritance

By default, a properties index is inherited in sub classes (i.e. if an attribute is indexed in a super class, it is also indexed in a sub class). If you need to change the index type of an attribute in a subclass you can override the attribute and annotate it with [SpaceIndex] using the requested index type (to disable indexing use NONE).

Basic Index

There are two basic index types provide:

  • BASIC index - this speeds up equality matching (equal to/not equal to).
  • EXTENDED index - this speeds up comparison matching (bigger than/less than).

Here is an example how you can define indexes:

using System;
using System.Collections.Generic;

using GigaSpaces.Core.Metadata;

	public class User {

		[SpaceID(AutoGenerate = false)]
		[SpaceRouting]
		public long? Id { set; get; }
		[SpaceIndex(Type = SpaceIndexType.Basic)]
		public String Name{ set; get; }
		public double? Balance{ set; get; }
		[SpaceIndex(Type = SpaceIndexType.Extended)]
		public double? CreditLimit{ set; get; }

        [SpaceProperty(StorageType = StorageType.Document)]
		public Nullable<EAccountStatus> Status{ set; get; }

        [SpaceProperty(StorageType = StorageType.Document)]
        [SpaceIndex(Path = "ZipCode", Type = SpaceIndexType.Basic)]
		public Address address{ set; get; }

		[SpaceIndex(Path = "[*]")]
		public String[] Comment{ set; get; }

        [SpaceProperty(StorageType = StorageType.Document)]
		[SpaceIndex(Path = "HOME")]
		public Dictionary<String, String> Contacts{ set; get; }

        [SpaceProperty(StorageType = StorageType.Document)]
		[SpaceIndex(Path = "[*].Id")]
		public List<Group> Groups{ set; get; }

        [SpaceProperty(StorageType = StorageType.Document)]
		[SpaceIndex(Path = "[*]")]
		public List<int?> Ratings{ set; get; }

	    //........
    }
}

Compound Indexing

A Compound Index is a Space index composed from several attributes or nested attributes. Each attribute of a compound index is called a segment and each segment is described by its path.

Here is an example of a compound index:

using System;
using System.Collections.Generic;

using GigaSpaces.Core.Metadata;

namespace xaptutorial.model
{
	[CompoundSpaceIndex(Paths = new[] {"Name", "CreditLimit"})]
	[SpaceClass]
	public class User {
		[SpaceID(AutoGenerate = false)]
		[SpaceRouting]
		public long? Id { set; get; }

		// ......
    }
}
// Here is a query that will use this index
SqlQuery<User> query = new SqlQuery<User>("Name = 'John Dow' AND CreditLimit > 1000");

There are several additional indexing options available. For example you can index nested attributes, Nested Dictionaries, Collections, nested attributes within a Collection, free text search and others.

See also:

Indexing Objects

Best Practice

When you code your Space classes make sure:
  • there are indexes for all relevant attributes including nested attributes you use for queries

  • numeric attribute queried with between / greater / less than should have an extended index.

  • compound indexes should be used for attributes queried using AND query

  • Space classes have empty no arg constructor

  • all nested classes are serializable

  • when possible use writeMultiple.

  • use projection for read/readMultiple

  • use clear for data removal and not take or takeMultiple

  • no huge collections with many items

  • use change api instead of update , especially if collections are used.