ulimit about the configuration of open files
The operating system can only open 1024 files by default. If the number of files opened exceeds this number, the program will have an error of "too many open files". 1024 is obviously not enough for big data systems. If not set, basically the entire big data system is "unavailable" and cannot be used in the production environment at all.
In centos 5/6 and other versions, the resource limit configuration can be set in /etc/security/, for each user such as root/user or * on behalf of all users.
Of course, it can be configured in /etc/security//. The system first loads the configuration files in the directory alphabetical order, and then loads the configuration to override the previous configuration.
A configuration example
as follows:
* soft nofile 65535 * hard nofile 65536 * soft nproc 65535 * hard nproc 65535 * soft core 65535 * hard core 65535
Configuration method
as follows:
echo "* soft nofile 65535" >> /etc/security/ echo "* hard nofile 65535" >> /etc/security/ echo "* soft nproc 65535" >> /etc/security/ echo "* hard nproc 65535" >> /etc/security/
In bash, there is an ulimit command that provides control of the shell and the available resource for the process started by the shell. It mainly includes the number of open file descriptors, the maximum number of users' processes, the size of coredump file, etc.
In CentOS 7 / RHEL 7 systems, Systemd is used instead of the previous SysV, so the configuration scope of the /etc/security/ file is slightly narrowed. The configuration here only applies to resource restrictions for users who log in with PAM authentication, and it does not take effect on systemd service's resource restrictions. The restrictions for login users are as mentioned above, just use /etc/security/ and .
Configuration of resource restrictions on systemd service
Global configuration, placed in the files /etc/systemd/ and /etc/systemd/.
At the same time, all .conf files in the two corresponding directories /etc/systemd//*.conf and /etc/systemd//*.conf will be loaded.
Among them, it is used by the system instance and used by the user instance. For general service, the configuration in use is enough.
The configuration in /*.conf will be overwritten.
The configuration parameters are as follows:
DefaultLimitCORE=infinity DefaultLimitNOFILE=65535 DefaultLimitNPROC=65535
Notice:
- After modification
- The system needs to be restarted before it will take effect
For a single service, it can also be set, taking nginx as an example.
Edit the /usr/lib/systemd/system/ file, or the /usr/lib/systemd/system// file
Make the following configuration:
[Service] LimitCORE=infinity LimitNOFILE=100000 LimitNPROC=100000
Then run the following command to take effect.
systemctl daemon-reload systemctl restart
Check out the limit settings of a process:
cat /proc/YOUR-PID/limits
For example, the configuration effect of one of my nginx service:
cat /proc/$(cat /var/run/)/limits
By the way:
CentOS7 comes with /etc/security//. The default setting is 4096 for non-root users. The configuration will be overwritten by the configuration in the directory. This requires attention. It needs to be modified for ordinary users.
Summarize
The above is personal experience. I hope you can give you a reference and I hope you can support me more.