File Descriptors

The GigaSpaces LRMI communication layer opens network connections dynamically. With large-scale applications, or with clients that are running a large number of threads accessing the data grid, you might end up having a large number of file descriptors used both on the client and server side. You may also have multiple JVMs running on the machine. If so, you may have to increase the default max user processes value.

For Linux environments, if you don't have the correct file descriptor settings, there will be Too many open files error messages in the GigaSpaces processes log files or on the client side. Low file descriptor values will affect application stability, along with the ability of clients to connect to the GigaSpaces in-memory data grid. Set both the System level File Descriptor Limit and the Process level File Descriptor Limit to high values to avoid these problems (see below for instructions).

System File Descriptor Limit

To change the number of file descriptors in Linux, as the root user edit the following line in the /etc/sysctl.conf file:

fs.file-max = 300000

300000 will be the new file descriptor limit that you want to set.

Apply the change by running the following command:

/sbin/sysctl -p

Verify your settings using:

cat /proc/sys/fs/file-max 

OR

sysctl fs.file-max

Process File Descriptor Limit

The Linux OS by default has a relatively small number of file descriptors available and max user processes (1024). You should make sure that your standalone clients, or GSA Grid Service Agent. This is a process manager that can spawn and manage Service Grid processes (Operating System level processes) such as The Grid Service Manager, The Grid Service Container, and The Lookup Service. Typically, the GSA is started with the hosting machine's startup. Using the agent, you can bootstrap the entire cluster very easily, and start and stop additional GSCs, GSMs and lookup services at will./GSM Grid Service Manager. This is is a service grid component that manages a set of Grid Service Containers (GSCs). A GSM has an API for deploying/undeploying Processing Units. When a GSM is instructed to deploy a Processing Unit, it finds an appropriate, available GSC and tells that GSC to run an instance of that Processing Unit. It then continuously monitors that Processing Unit instance to verify that it is alive, and that the SLA is not breached./GSC Grid Service Container. This provides an isolated runtime for one (or more) processing unit (PU) instance and exposes its state to the GSM./LUS Lookup Service. This service provides a mechanism for services to discover each other. Each service can query the lookup service for other services, and register itself in the lookup service so other services may find it. using a user account which have its maximum open file descriptors (open files) and max user processes configured to a high value. A good value is 32768.

Setting the max open file descriptors and max user processes is done via the following call:

ulimit -n 32768 -u 32768

Alternatively, you should have the following files updated:

/etc/security/limits.conf

- soft    nofile          32768
- hard    nofile          32768

/etc/security/limits.d/90-nproc.conf

- soft nproc 32768

Monitoring Utilized File Descriptors

You can monitor the MaxFileDescriptorCount and OpenFileDescriptorCount using the JConsole: