Tiered Storage

Tiered Storage provides seamless tiering of data between different storage tiers, to balance performance and storage media costs..

Data can be stored in two tiers:

  • Hot Tier — RAM: Priority data, used for read/write operations and analysis requiring fastest access

  • Warm Tier — SSD (recommended media): Additional data used mostly for read-only, analytical purposes

By defining business rules, you direct the system which data to store in the Hot tier and which in the Warm tier. The Hot/Warm decision can be based on the data content, or by data aging considerations.

Using Tiered Storage can increase effective data capacity by a factor of ten.

Optimizing the Use of Tiered Storage

In order to get the most out of this feature, please note the following:

  • Data in the Hot tier is also stored in the Warm tier (except for transient objects — see below).

  • Queries use the Hot tier if all of the required data is in the Hot tier.

  • Primitive property types and selected Java classes are supported in Tiered Storage. See Tiered Storage Date Types for a complete list of supported date types.

For tables configured as Tiered Storage (see below), if no rules are defined for the table, its data will be stored in the Warm tier only.

For ServiceGrid SLA, the Processing Unit must specify the instance level SLA with host requirement for each instance or alternatively, specify pu.autogenerated-instance-sla=true as a context property of the deployment. This will allow the system to automatically generate the SLA .

For ServiceGrid, each instance is bound to a specific machine for the entire duration of the deployment. Redeployment is only possible with the same schema, unless delete was called.

How to Configure Tiered Storage in XML

Tiered Storage is implemented by parameters coded in the XML file for the Space. The relevant sections of the XML file are shown below.

Sample XML Code

The following XML specifies that for the table named Order, the orderTime field will be compared to the current time. For records where the orderTime field is within 24 hours of the current time, the record will be in Hot storage and Warm storage. After that time, it will be in Warm storage only.

        
<os-core:tiered-storage>
 <os-core:tables>
 <os-core:table name="com.mycompany.app.model.Order" time-column="orderTime">
 <os-core:cache-rule period="PT24H"/>
 </os-core:table>
 </os-core:tables>
</os-core:tiered-storage>

 

The following XML specifies that for the table named Customer, the club field will be compared to the string "PLATINUM". Matching records be in Hot storage and Warm storage. Other records will be in Warm storage only.

It also specifies that for the TransientCustomer table, all data will be stored in Hot storage only.

              
<os-core:tiered-storage>
 <os-core:tables>
 <os-core:table name="com.mycompany.app.model.Customer">
 <os-core:cache-rule criteria="club = 'PLATINUM'"/>
 </os-core:table>
 <os-core:table name="com.mycompany.app.model.TransientCustomer" transient="true"/>
 </os-core:tables>
</os-core:tiered-storage>

 

Following is a summary of XML parameters.

XML Parameter Meaning
tiered-storage Beginning of Tiered Storage definitions for a Space
tables Beginning of list of tables
table name=name-of-table Tiered rule for a table (object type)
time-column=name-of-field Name of the field (property) in the table that will be evaluated for eviction from RAM. Must be of Java type TimeStamp, Long or Instant.
period=period After the period amount of time has passed, it will be removed from Hot storage (RAM).
period must be a format supported by the Java Duration class.
Examples:
PT15M (15 minutes)
PT20S (20 seconds)
cache-rule criteria="fieldname operator value"

Rule to determine which data to place in Hot storage.
fieldname can be any Java primitive field type.
operator can be =   !=   >=   <=
value is a compatible value for comparision to the field.

Examples:

cache-rule criteria="club! = 'BASIC'"

cache-rule criteria="active = true"

cache-rule criteria="age >=30'"

cache-rule criteria="all" Place all data in both Hot and Warm storage
transient=true|false Specify if the data is in the Hot tier only. Default is transient=false.

How to Configure Tiered Storage in Java Code

Tiered Storage is implemented by setting parameter values in the Java code definition for the Space. Relevant parts of the Java code are shown below.

Sample Java Code

The following Java code specifies that for the table named Customer, the club field will be compared to the string "PLATINUM". Matching records will be stored in Hot storage and Warm storage. Other records will be in Warm storage only.

It also specifies that for the TransientCustomer table, all data will be stored in Hot storage only.

In addition, records in the Purchase table will be stored in Hot and Warm storage for 30 seconds. After that time, they will be stored in Warm storage only.

 @Override 
protected void configure(EmbeddedSpaceFactoryBean factoryBean) {
 super.configure(factoryBean); 
 TieredStorageConfig tieredStorageConfig = new TieredStorageConfig(); 
 Map<String, TieredStorageTableConfig> tables = new HashMap<>();
 tables.put(Customer.class.getName(), new TieredStorageTableConfig().sethame(Customer.class.getName()).setCriteria("club =  'PLATINUM'")); 
 tables.put(TransientCustomer.class.getName(), new TieredStorageTableConfig().setName(TransientCustomer.class.getName()).setTransient(true)); 
 tables.put(Purchase.class.getName(), new TieredStorageTableConfig.setName(Purchase.class.getName().setTimeColumn("orderTime").setPeriod(Duration.ofSeconds(30)));
 tieredStorageConfig.setTables(tables); 
 factoryBean.setTieredStorageConfig(tieredStorageConfig); 

 

Following is a summary of the relevant Java code.

Code Meaning
Customer.class.getName()).setCriteria("club = 'PLATINUM'")

Rule to determine which data to place in Hot storage.
In this example, rows of data in the Customer table are placed in Hot storage is the club field is equal to "PLATINUM".

Note that the operator can be =   !=   >=   <=

TransientCustomer.class.getName()).setTransient(true) Rule to determine which data is in Hot storage only.
In this example, all data in the TransientCustomer table is stored in Hot storage, but not in Warm storage.
Purchase.class.getName().setTimeColumn("orderTime")
.setPeriod(Duration.ofSeconds(30))
Rule to specify the amount of time that certain data will be in Hot storage.
In this example, for the Purchase table, the orderTime fields will be kept in Hot storage for 30 seconds.