This page describes an older version of the product. The latest stable version is 16.4.

Common User Issues


Num. Problem Troubleshooting Information & Possible causes Logging information needed by GigaSpaces support
1. Deployment failure * Validate if the GSC’s with appropriate SLA definitions are started.
- Validate if GSC’s are registered to the same lookup locators with same lookup group.
- Validate if all the dependencies (classes and jars) for the processing units are included in the deployed jar/war.
* Server side logs (all GSC’s, GSM’s, LUS’s logs and console logs for each)
2. Client cannot connect to space * Validate if Space has been successfully deployed.
- Validate if the client URL corresponds to the space that is has been deployed (host/ip, lookup group(s), space name).
- Validate if Space process did not crash because of some error(s).
* Client side logs.
Server side logs (all GSC’s, GSM’s, LUS’s logs and console logs for each).
3. Memory shortage on space * Validate if space has more data than expected.
- If space is having lot of data, unwanted data should be evicted from space. More memory should be allocated to the cluster members to avoid getting into the situation (check XAP memory manager watermarks).
- If not a lot of data, it could be a JVM garbage collection issue.
- Validate if this is a replication issue, is the backup partition terminated? Are primary and backup disconnected which is causing replication redo log size to grow. Restore the backup space partition for redo log to clear and make room for new data.
- Validate if this is a mirror replication issue - is the mirror terminated? Are primary space and mirror disconnected which is causing redo log size to grow. Restore the backup space partition for redo log to clear and make room for new data.
* Server side logs (all GSC’s, GSM’s, LUS’s logs and console logs for each).
- JVM GC logs for space partitions (GSC’s that are hosting spaces). * Heap dump of the primary space partition (GSC hosting this space).
4. Space remote client queries (reads/writes) taking longer than usual * Validate if the object has appropriate indexes, proper serialization.
- Validate if query criterion is different from what is expected, causing larger result set than usual and slowing the queries.
- Validate if object has a payload that is larger than normal.
- Validate if it is a lrmi thread pool issue where lot of concurrent clients are accessing the space and no more threads are left on the space to accept new client connections.
- Validate if the client broadcast thread pool is fully exhausted.
- Validate if backup space or mirror are having any problems which could lead to slower query performance on space.
- Validate if it is a temporary issue caused by JVM garbage collection.
* Client side logs.
- JVM GC logs for space partitions (GSC’s that are hosting spaces).
- Run Thread dumps for the space partitions (GSC’s that are hosting spaces) 2-3 times with 1 minute gap and include logs for space partitions.
5. Space remote client execute queries taking longer than usual * Validate if query criterion is different from what is expected, causing larger result set than usual and slowing the execute requests.
- Validate if object has a payload that is larger than normal.
- Validate if it is a lrmi thread pool issue where lot of concurrent clients are accessing the space and thread pool is exhausted.
- Validate if the client broadcast thread pool is fully exhausted.
- Validate if backup space or mirror are having any problems which could lead to slower remoting performance on space.
- Validate if it is a temporary issue caused by JVM garbage collection.
* Client side logs.
- JVM GC logs for space partitions (GSC’s that are hosting spaces).
- Run Thread dumps for the space partitions (GSC’s that are hosting spaces) 2-3 times with 1 minute gap and include logs for space partitions.
6. Notification delivery is slower than usual * Validate if this is a slow consumer issue. Make sure the server and client are using appropriate slow consumer configuration settings
- Validate if notification batch size is large and more data than expected matches the notification template that was defined.
- Validate if it s a temporary issue caused by JVM garbage collection.
- Validate if it is a lrmi thread pool issue where lot of clients are registered for notifications for the same data and thread pool is exhausted.
* Client side logs.
- JVM GC logs for space partitions (GSC’s that are hosting spaces).
- Run Thread dumps for the space partitions (GSC’s that are hosting spaces) 2-3 times with 1 minute gap and include logs for space partitions.
- GS-UI statistics screen shots for the space partitions. You will see the difference between Notifications sent and Notifications ack counts.
7. Wrong clustered space usage SEVERE error on the server side * This error message indicates that there was a problem replicating a certain object to the target space. The problem could be caused by several reasons:
- The replicated object no longer exists in the target space(in case it was updated, taken or deleted).
- The object already exists in the target space(in case it was written for the first time and an object with the same id already exists in the target space).
- The object version in the target space is different than the one in the source space (in case it was updated).
8. Replication Redo log capacity exceeded errors * Validate if the backup partition is terminated? Restore the backup space partition for redo log to clear and make room for new data.
- Validate if primary and backup members are disconnected because of a networking issue? Causing redo log size to grow. Restore the network and restart the backup member in order to recover the current data from primary partition.
- Backup member going thru a JVM garbage collection causing pause for seconds while the primary space has lot of updates (not very common scenario).
* Client side logs.
- Server side logs (all GSC’s, GSM’s, LUS’s logs and console logs for each).
- JVM GC logs for space partitions (GSC’s that are hosting spaces).
9. Replication Redo log overflows to disk This happens when redo log memory capacity is exceeded and GigaSpaces starts writing redo logs to disk. * Validate if the backup partition is terminated? Restore the backup space partition for redo log to clear and make room for new data.
- Validate if primary and backup members are disconnected because of a networking issue? Causing redo log size to grow. Restore the network and restart the backup member in order to recover the current data from primary partition.
- Backup member going thru a JVM garbage collection causing pause for seconds while the primary space has lot of updates (not very common scenario).
* Client side logs.
- Server side logs (all GSC’s, GSM’s, LUS’s logs and console logs for each).
- JVM GC logs for space partitions (GSC’s that are hosting spaces).
10. Missing partitions or instances of GigaSpaces application * Validate if any machine that hosts the GigaSpaces cluster crashed. Identify the root cause for this and restore the machine.
- Validate if any of the GSC JVM’s failed with a JVM hotspot error.
* JVM typically creates a file beginning with hs_err. This file has the cause of error.
- Server side logs (all GSC’s, GSM’s, LUS’s logs and console logs for each).
- JVM GC logs for space partition/instance (GSC) that had the error.
11. Provision failure (planned > actual) - The processing unit has less actual instances than planned instances * Validate if any machine that hosts the GigaSpaces cluster crashed. Identify the root cause for this and restore the machine.
- Validate if any of the GSC JVM’s failed with a JVM hotspot error.
* JVM typically creates a file beginning with hs_err. This file has the cause of error.
- Server side logs (all GSC’s, GSM’s, LUS’s logs and console logs for each).
- JVM GC logs for space partition/instance (GSC) that had the error.