Overview


The purpose of this feature is to add support for “Quiesce state” (a.k.a maintenance mode) for a processing unit. If a space is Quiesced, it rejects all data-related operations with an exception, allowing the user to perform maintenance operations.

Regardless to the Space, the XAP Polling Container and Notify Container are also aware of quiesce state and will handle quiesce state changed events by implementing QuiesceStateChangedListener.
As well as XAP event containers mentioned above, a user defined bean can also handle quiesce state changed events by implementing QuiesceStateChangedListener.
Quiesce is mostly triggered using ProcessingUnit API and propagated to all space instances and quiesce state changed listeners in the processing unit.

See also:

Quiesce mode can be also triggered using CLI (see Quiesce CLI)

Motivation

There are several user stories which can benefit from quiesce state such as:

Safe shutdown

Shutdown an application which contains a data grid and a mirror in a safe manner - make sure all operations completed on the grid are flushed to the mirror before undeploy:

  1. Enter quiesce mode.
  2. Wait for replication between the space and the mirror to complete (by monitoring the redolog).
  3. Undeploy the space and mirror.

Rolling system upgrade

Perform a rolling system upgrade on a live system (typically a processing unit code upgrade) and avoid inconsistencies related to processing of requests during the upgrade process:

  1. Enter quiesce mode
  2. Perform the hot-upgrade pattern:
    2.1. Update the pu in the GSM
    2.2. Restart the backups
    2.3. Restart the primaries
  3. Exit quiesce mode - the upgraded system resumes and works as expected
See also:

Here you can find use cases examples

Space Behavior During Quiesce Mode

  • The space will reject all user data-related new operations by throwing QuiesceException back to the user, replication of previous on going operations continues.
  • Commit and abort are allowed in case of two phase commit transactions.
  • Commit and prepare of a single phase transactions will be rejected.
  • Lease manager will be paused except extend lease operation in order to prevent losing notify listener templates.

Security

  • On a secured system, one needs a Manage PU privileges to quiesce/unquiesce a processing unit.

Configuration

  • This feature can be disabled by setting com.gs.engine.disableQuiesceMode to true.

Limitations

The following limitations and open issues apply to Quiesce Mode:

  • The processing unit should be intact before triggering quiesce request.
  • Though Quiesce handles processing unit restart, it is not resilient to sudden network disconnections (in XAP grid components machines) therefore it is possible that in rare conditions the quiesce request will have to be repeated manually by the user.
  • Quiesce state changed events are propagated only to the components (beans) that are located within the processing unit context file (pu.xml), therefore custom components will not be aware of quiesce state changed events (even if the component implements QuiesceStateChangedListener).
  • Replication of Quiesce state between primary and backup GSM is not yet supported - GSM failover may cause loosing the quiesce state of the system, in this case repeating quiesce request will also be required in order to re inform the GSM about quiesce state.