Summary: Working with Space Documents
OverviewThe GigaSpaces document API exposes the space as Document Store. A document, which is represented by the class SpaceDocument, is essentially a collection of key-value pairs, where the keys are strings and the values are primitives, String, Date, other documents, or collections thereof. Most importantly, the Space is aware of the internal structure of a document, and thus can index document properties at any nesting level and expose rich query semantics for retrieving documents. Unlike concrete objects, which force users to design a fixed data schema (in the form of a class definition) and adhere to it, a document is much more dynamic - users can add and remove properties at runtime as necessary. A Document always belongs to a certain type, represented by the interface ISpaceTypeDescriptor. Before a certain Document instance is written to the space, its type should be introduced to the data grid. The type has a name and controls metadata such as an identifier property, a routing property and which properties are initially indexed (naturally, you can also index new properties at runtime after adding them to your documents).
Note that the Document type does not describe the properties themselves (except for the names of the ID and Routing properties). These are completely dynamic and each instance can have a different set of properties (although in most cases Document instances of the same type are likely to have identical or similar set of properties). Schema Evolution with Space DocumentsSince a SpaceDocument is completely dynamic by nature, it is very easy to change or evolve your data model without ever taking down the Space. Simply change your application code to add additional properties or remove existing ones, and you're good to go. Even better, old and new versions can co-exist since the space does not enforce any restriction with regards to the property set of documents that belong to a certain type. This is a much more lightweight model in comparison to the classic concrete object model, where a recompilation and in many cases a full space restart is required to change the data schema. Type DefinitionBefore we start writing and reading SpaceDocument from the space, we need an initial schema definition of the document type. For example, suppose we're implementing an electronic commerce system, and decided we need a type called Product with the following properties:
We also decide that CatalogNumber will be a primary key, partitioning will be done by the Category property, and the properties Name, Price should be indexed since they participate in most of the queries executed. Remember, the type definition is for metadata only, so we're not concerned about Description and other such fields in the type definition, because Description isn't used for indexing or any other metadata. The following is an example of how to introduce a new document type: ISpaceProxy spaceProxy = ... //Obtain a space proxy // Create type descriptor: SpaceTypeDescriptorBuilder typeBuilder = new SpaceTypeDescriptorBuilder("Product"); typeBuilder.SetIdProperty("CatalogNumber"); typeBuilder.SetRoutingProperty("Catagory"); typeBuilder.AddPathIndex("Name"); typeBuilder.AddPathIndex("Price", SpaceIndexType.Extended); ISpaceTypeDescriptor typeDescriptor = typeBuilder.Create(); // Register type descriptor: spaceProxy.TypeManager.RegisterTypeDescriptor(typeDescriptor); Note that this code does not reflect the complete model - most of the properties do not need to be introduced to the schema. Only properties with special roles (ID, Routing) are part of the schema definition. These meta model settings cannot be changed without restarting the space or dropping the type, clearing all its instances and reintroducing it again. Creating and Writing DocumentsTo create a document in the space, create a SpaceDocument object using the type name, and set the desired properties. Then write it to the space using the regular ISpaceProxy write method: public void WriteProduct1(ISpaceProxy spaceProxy) { // 1. Create the document using the type name SpaceDocument document = new SpaceDocument("Product"); // 2. Create the properties: document["CatalogNumber"] = "hw-1234"; document["Category"] = "Hardware"; document["Name"] = "Anvil"; document["Price"] = 9.99f; document["Tags"] = new String[] {"heavy", "anvil"}; IDictionary<String, Object> features = new Dictionary<String, Object>(); features["Manufacturer"] = "Acme"; features["RequiresAssembly"] = false; features["Weight"] = 100; document["Features"] = features; IDictionary<String, Object> review1 = new Dictionary<String, Object>(); review1["Name"] = "Wile E. Coyote"; review1["Rate"] = 1; review1["Comments"] "Don't drop this on your toe, it will hurt."; IDictionary<String, Object>> review2 = new Dictionary<String, Object>(); review2["Name"] = "Road Runner"; review2["Rate"] = 5; review2["Comments"] = "Beep Beep!"; document["Reviews"] = new IDictionary<String, Object>[] {review1, review2}); // 3. Write the document to the space: spaceProxy.write(document); } Using generic types as nested properties (i.e IDictionary<String, Object> features) will be read as IDictionary<Object, Object> when the above document is read from the space. There are two ways to receive back a dictionary of the same generic type. One is using the DocumentProperties class instead, and the other is to use a BinaryCustom storage type for the dynamic properties. The DocumentProperties class extends IDictionary<String, Object> and besides being strongly typed to String keys and Object values, it provides better performance when used. public void WriteProduct2(ISpaceProxy spaceProxy) { // 1. Create the document using the type name SpaceDocument document = new SpaceDocument("Product"); // 2. Create the properties: document["CatalogNumber"] = "hw-1234"; document["Category"] = "Hardware"; document["Name"] = "Anvil"; document["Price"] = 9.99f; document["Tags"] = new String[] {"heavy", "anvil"}; DocumentProperties features = new DocumentProperties(); features["Manufacturer"] = "Acme"; features["RequiresAssembly"] = false; features["Weight"] = 100; document["Features"] = features; DocumentProperties review1 = new DocumentProperties(); review1["Name"] = "Wile E. Coyote"; review1["Rate"] = 1; review1["Comments"] "Don't drop this on your toe, it will hurt."; DocumentProperties review2 = new DocumentProperties(); review2["Name"] = "Road Runner"; review2["Rate"] = 5; review2["Comments"] = "Beep Beep!"; document["Reviews"] = new DocumentProperties[] {review1, review2}); // 3. Write the document to the space: spaceProxy.write(document); }
Reading and Removing DocumentsThere are three types of document queries: Template QueryThis type of query uses a SpaceDocument with type and any other set of properties values as a template for the query public SpaceDocument ReadProductByTemplate(ISpaceProxy spaceProxy) { // Create template: SpaceDocument template = new SpaceDocument("Product"); template["Name"] = "Anvil"; // Read: SpaceDocument result = spaceProxy.Read(template); return result; } Sql QueryYou can use the SqlQuery to search for matching SpaceDocument entries. For example: to read a document of type Product whose Price is greater than 15: public SpaceDocument ReadProductBySql(ISpaceProxy spaceProxy) { // Create query: SqlQuery<SpaceDocument> query = new SqlQuery<SpaceDocument>("Product", "Price > ?"); query.SetParameter(1, 15f); // Read: SpaceDocument result = spaceProxy.Read(query); return result; }
Queries on nested properties are supported. For example, to read products manufactured by Acme: public SpaceDocument[] ReadProductBySqlNested(ISpaceProxy spaceProxy) { // Create query: SqlQuery<SpaceDocument> query = new SqlQuery<SpaceDocument>( "Product", "Features.Manufacturer = ?"); query.SetParameter(1, "Acme"); // Read: SpaceDocument[] result = spaceProxy.ReadMultiple(query, 10); return result; } ID Based QueryFor example: Read a document of type Product whose ID is hw-1234: public SpaceDocument ReadProductById(ISpaceProxy spaceProxy) { return spaceProxy.ReadById(new IdQuery<SpaceDocument>("Product", "hw-1234")); } Queries by multiple Ids are supported. For example: public SpaceDocument[] ReadProductByMultipleIds(ISpaceProxy spaceProxy) { Object[] ids = new Object[] {"hw-1234", "av-9876"}; IReadByIdsResult<SpaceDocument> result = spaceProxy.ReadByIds(new IdsQuery<SpaceDocument>("Product", ids)); return result.ResultsArray; }
Nested PropertiesThe Document properties' values can be either scalars (integers, strings, enumerations, etc), collections (arrays, lists), or nested properties (dictionary or an extension of dictionary, such as DocumentProperties). Values must adhere to the same restrictions as in the concrete object model (e.g. be serializable). Nested properties can be queried by using the dot ('.') notation to describe paths, as shown above.
IndexingProperties and nested paths can be indexed to boost queries performance. In the type registration sample above, the Name and Price properties are indexed. Since the schema is flexible and new properties might be added after the type has been registered, it is possible to add indexes dynamically as well.
EventsEvent containers (both polling container and notify container) support SpaceDocument entries. Here is a simple example of a polling event container configuration using a SpaceDocument: [PollingEventDriven] public class SimpleListener { [EventTemplate] public Data UnprocessedData { get { SpaceDocument template = new SpaceDocument("Product"); template["Name"] = "Anvil"; return template; } } [DataEventHandler] public SpaceDocument EventListener(SpaceDocument event) { //process Data here } } FIFOFIFO support is off by default with SpaceDocument entries (as with concrete object entries). To enable FIFO support, modify the type introduction code and set the desired FIFO support mode. For example: // Create type descriptor: ISpaceProxy spaceProxy = ... //Obtain a space proxy // Create type descriptor: SpaceTypeDescriptorBuilder typeBuilder = new SpaceTypeDescriptorBuilder("Product"); // Other type descriptor settings. typeBuilder.FifoSupport = FifoSupport.Operation; ISpaceTypeDescriptor typeDescriptor = typeBuilder.Create(); // Register type descriptor: spaceProxy.TypeManager.RegisterTypeDescriptor(typeDescriptor);
Transactions and Optimistic LockingTransactions and isolation modifier semantics is identical to the concrete object semantics. Optimistic locking is disabled by default with SpaceDocument entries (same as with concrete object). To enable it, modify the type introduction code and set the optimistic locking support. For example: // Create type descriptor: ISpaceProxy spaceProxy = ... //Obtain a space proxy // Create type descriptor: SpaceTypeDescriptorBuilder typeBuilder = new SpaceTypeDescriptorBuilder("Product"); // Other type descriptor settings. typeBuilder.SupportsOptimisticLocking = true; ISpaceTypeDescriptor typeDescriptor = typeBuilder.Create(); // Register type descriptor: spaceProxy.TypeManager.RegisterTypeDescriptor(typeDescriptor);
Dynamic Properties Storage TypeBy default, the dynamic properties of the document will be deserialized fully and kept in space as an instance of their corresponding space type class. This allows indexing and nested matching against these properties. However, in some scenarios, the data kept inside the dynamic properties is not used for matching. In that case it is possible to keep these properties serialized in their binary format in the space, increasing performance and reducing memory consumption. However, the data stored in binary format will be not be visible via the management center or readable from an interoperable Java/C++ client, it will appear as binary large object to these clients and will only deserialized when a .NET client reads it. The way to specify the storage type is when constructing the type using the SpaceTypeDescriptorBuilder: // Create type descriptor: ISpaceProxy spaceProxy = ... //Obtain a space proxy // Create type descriptor: SpaceTypeDescriptorBuilder typeBuilder = new SpaceTypeDescriptorBuilder("Product"); // Other type descriptor settings. typeBuilder.DynamicPropertiesSupport(true, StorageType.Binary); ISpaceTypeDescriptor typeDescriptor = typeBuilder.Create(); // Register type descriptor: spaceProxy.TypeManager.RegisterTypeDescriptor(typeDescriptor);
PersistencyThe External Data Source is supported for space documents, however, if the EDS is also used to initially load the space with data, the space needs to be aware of the type descriptors of the documents. Therefore, the space needs to be started with known type descriptors. Example on how to implement an EDS that persists SpaceDocuments of type "Trade" and start a space in a basic processing unit container with the EDS and known Trade type descriptors:
Starting the Space
[ContainerInitializing] public void Initialize(BasicProcessingUnitContainer container) { //Create a new space configuration object that is used to start a space SpaceConfig spaceConfig = new SpaceConfig(); //Set a new ExternalDataSource config object spaceConfig.ExternalDataSourceConfig = new ExternalDataSourceConfig(); //Configure EDS related stuff ... spaceConfig.ExternalDataSourceConfig.Instance = new DocumentEDS(); //Set the EDS to work with documents spaceConfig.ExternalDataSourceConfig.EntryType = EntryType.Document; SpaceTypeDescriptor tradeTypeDescBuilder = new SpaceTypeDescriptor("Trade"); //Set id property with auto generate true tradeTypeDescBuilder.SetIdProperty("Uid", true); tradeTypeDescBuilder.SetRoutingProperty("SymbolLabel"); //Adding type descriptor to the space known type descriptors spaceConfig.SpaceTypeDescriptors.Add(tradeTypeDescBuilder.Create()); //Create the space ISpaceProxy spaceProxy = container.CreateSpaceProxy( "MySpace", "/./MySpace", spaceConfig); } The EDS Implementation public class DocumentEDS : ISqlDataSource { public void Init(Dictionary<String, String> properties) { // initialize persistency layer } IDataEnumerator InitialLoad() { // load all the data from persistency // build and return an iterator of documents } public void ExecuteBulk(IList<BulkItem> bulk) { foreach (BulkItem bulkItem in bulk) { SpaceDocument document = (SpaceDocument) bulkItem.Item; switch (bulkItem.Operation) { case BulkItem.WRITE: // WriteDocument(document); break; case BulkItem.UPDATE: // UpdateDocument(document); break; case BulkItem.REMOVE: //RemoveDocument(document); break; default: break; } } } public void Shutdown() { //cleanup resources and close the persistency } } Concrete objects can be persisted via document EDS as well, in the same way.
Advanced Options |
![]() |
GigaSpaces.com - Legal Notice - 3rd Party Licenses - Site Map - API Docs - Forum - Downloads - Blog - White Papers - Contact Tech Writing - Gen. by Atlassian Confluence |