Storage Optimization

Storage Optimization is a feature of GigaSpaces that reduces the memory footprint (RAM) of the the properties of a SpaceClosed Where GigaSpaces data is stored. It is the logical cache that holds data objects in memory and might also hold them in layered in tiering. Data is hosted from multiple SoRs, consolidated as a unified data model. Object, with minimal impact on system performance.

How Does Storage Optimization Work?

In order to reduce the system's RAM footprint, storage-optimized properties are stored and packed together in a binary format. This reduces the amount of memory required for the properties.

How to Decide Whether to Use Storage Optimization?

The decision whether or not to use Storage Optimization is similar to how you decide whether to index a property.

The reduced memory footprint achieved with Storage Optimization is most effective if the Space Object has a high proportion of non-indexed properties that are used primarily in read requests. Properties that are used in queries as keys or aggregated values may not benefit as much from Storage Optimization, due to the frequency of serialization-deserialization that is required.

How to Configure Storage Optimization?

Storage Optimization can be configured when the data is modeled using the Ops Manager user interface, or by specifying annotations in the Java code.

Configure Storage Optimization via Ops Manager

In Ops Manager one-click connect, a toggle on the top of the screen lets you specify Storage Optimization for the Space Object:

Clicking on the toggle will select Storage Optimization for every non-indexed property in the object. This covers the vast majority of the cases. After loading and measuring the data capacity, toggling of individual properties to be BINARY or COMPRESSED is available via the API.

Configure Storage Optimization via the API

Storage Optimization Options: Direct Layout and Sequential Layout

Two methods are available for Storage Optimization: the default of Direct Layout, and Sequential Layout.

With Direct layout, the header of the binary data includes indexes to allow direct access of each property. Direct layout reduces the CPU overhead of Storage Optimization, with a small increase in the memory footprint.

The memory overhead for Direct layout is two bytes per indexed property. In this example, the memory overhead is 2 * 3 = six bytes.

If the value of a property is not set (null, or zero for primitive types), the memory overhead will be two bytes for the index, but no space will be used for the property data.

Direct layout is limited to a maximum of 64K bytes of data per object entry.

With Sequential layout, the properties are stored in binary format, without any indexing mechanism. This means that, for example, in order to locate the third optimized property, the system must first pass through the first and second optimized properties. Sequential layout minimizes the memory footprint, with some increase in CPU overhead, depending on the application's data access patterns.

The memory overhead for Sequential layout is one byte for each group of eight indexed properties. In this example, the memory overhead for three properties is one byte.

Invoke Storage Optimization for a Class

Storage Optimization Mode must be defined on the Object (class) level.

Storage Optimization is specified at the class level with either Direct (Selective) optimization (the default), or Sequential (Non-Selective) optimization.

Example for POJOClosed Plain Old Java Object. A regular Java object with no special restrictions other than those forced by the Java Language Specification and does not require any classpath.:

			
@SpaceClass
@SpaceClassBinaryStorage(layout = ClassBinaryStorageLayout.DIRECT)
public class TestObject {
    …
}

			
@SpaceClass
@SpaceClassBinaryStorage(layout = ClassBinaryStorageLayout.SEQUENTIAL)
public class TestObject {
…
}

Example for SpaceDocument:

			
SpaceTypeDescriptor typeDescriptorDoc1 = new SpaceTypeDescriptorBuilder("Doc1").supportsDynamicProperties(false)
.idProperty("id")
.addFixedProperty("info", String.class, StorageType.OBJECT)
.addFixedProperty("type", String.class)
.addFixedProperty("id", Integer.class)
.binaryStorage(ClassBinaryStorageLayout.DIRECT).create();
					
			
SpaceTypeDescriptor typeDescriptorDoc1 = new SpaceTypeDescriptorBuilder("Doc1").supportsDynamicProperties(false)
.idProperty("id")
.addFixedProperty("info", String.class, StorageType.OBJECT)
.addFixedProperty("type", String.class)
.addFixedProperty("id", Integer.class)
.binaryStorage(ClassBinaryStorageLayout.SEQUENTIAL).create();

Property Level Annotation

Three options are available for Property annotation:

  • No storage optimization — StorageType.OBJECT

  • Storage optimization — StorageType.BINARY

  • Storage compression — StorageType.COMPRESSED
    This option uses bitwise manipulation to store smaller values in fewer bytes.

    • Very large values for the data type may require more bytes, e.g. a large long property may require 9 bytes instead of 8 bytes.

    • String types and user defined serializable types will be compressed using zip. This is usually effective for large values only.

By default, specifying Storage Optimization for the class sets StorageType.BINARY (storage optimization) for all non-indexed properties in the class, and StorageType.OBJECT (no storage optimization) for indexed items. This default can be overridden for individual properties, as follows.

		
@SpaceClass
@SpaceClassBinaryStorage
public class TestObject {
    …
    @SpacePropertyStorage(StorageType.OBJECT)
    public String getLastName() {
       return lastName;
    }
    ...
}

		
@SpaceClass
@SpaceClassBinaryStorage
public class TestObject {
    …
    @SpacePropertyStorage(StorageType.BINARY)
    public String getLastName() {
       return lastName;
    }
    ...
}

		
@SpaceClass
@SpaceClassBinaryStorage
public class TestObject {
    …
    @SpacePropertyStorage(StorageType.COMPRESSED)
    public String getLastName() {
       return lastName;
    }
    ...
}

Viewing Storage Optimization Settings in the Object Screen

Using the GigaSpaces Ops Manager, you can view the status of a Space Object. The Storage Optimized icon is displayed next to fields that are using Storage Optimization.

Limitations