Linux server settings for high load systems

There are a few basic settings you have to adjust for high load systems to make sure the server have enough resources to handle big number of network connections. The main parameter is a maximum number of opened files allowed for the process to keep at the same time. Each network connection uses a file handler therefore if the limit is too low you can quickly run out of handlers and the server can not accept any more connections.

This limit is set on 2 levels – on the kernel level (fs.file-max) and on the system level (nofile). Another kernel property which can be important in certain configurations (like transports installations or when you use proxy for Bosh connections) is: net.ipv4.ip_local_port_range. This parameter can be set the same way as the fs.file-max property.

The fs.file-max kernel property is set via sysctl command. You can see current settings executing command:

view source print?

If you plan to run high load service with big number of server connections then this parameter should be at least as twice big as the number of network connections you expect to support. You can change this setting executing command:

view source

print?

You can see current settings executing command:

view source

print?

You can change this setting executing command:

view source

print?

Above commands let the system remember new settings until the next system restart. If you want to make the change permanent you have to edit file: /etc/sysctl.conf and add the property at the end of the file:

view source

print?

It will be automatically loaded next time you start the server. Command:

view source

print?

Causes the /etc/systcl.conf to be reloaded which is useful when you added more parameters to the file and don’t want to restart the server.

nofile

This is the property used by the system limits. For example running the command ulimit -a shows you all limits set for the current user:

view source

print?

To make it even more interesting and more complex there are 2 types of system limits: soft limit which can be temporarily exceeded by the user and hard limit which can not be exceeded. To see your hard limit execute command:

view source

print?

The hard limits are usually bigger then the soft limits or sometimes the same. For us the most important parameter is: open files. You can change the property in file: /etc/security/limits.conf. You have to append 2 following lines to the end of the file:

view source

print?

Where the jabber is the user name of the account running you IM service. You can also set the limits for all users on the machine in a following way:

view source

print?

For those changes to make an effect you have to logout from the modified account and login again. New limits should be applied.