I/O Wait more than 20%

[ericssonvietnam]

Those messages in the Apache logs are only PHP Notices and they can be ignored.
I know it is confusing that the line says error but they are truly only Notices.

[root@nagiosxi ~]# free -m
total used free shared buffers cached
Mem: 31941 30663 1277 1 2483 23467
-/+ buffers/cache: 4713 27227
Swap: 4095 208 3887

Above are the current resources on my server will creating swap file help in increasing the performance.

[ericssonvietnam]

Those messages in the Apache logs are only PHP Notices and they can be ignored.
I know it is confusing that the line says error but they are truly only Notices.

Can i create a swap file in order to have some performance enhancement

[WillemDH]

Talk to the person who setup the system and they should be able to provide the details needed.

IOWait generally means disk io related issues. Please take the time to find out on what type of disks your Nagios XI server is running and let us know.

[tgriep]

Your system already has a swap file installed on it.

[ericssonvietnam]

Those messages in the Apache logs are only PHP Notices and they can be ignored.
I know it is confusing that the line says error but they are truly only Notices.

After increasing the mentioned parameters CPU Usage is normal but still load average is high on the server as shown in below output :

top - 14:35:26 up 12:53, 2 users, load average: 13.54, 13.40, 13.31
Tasks: 320 total, 1 running, 318 sleeping, 0 stopped, 1 zombie
Cpu(s): 18.2%us, 10.8%sy, 0.0%ni, 35.2%id, 35.8%wa, 0.0%hi, 0.1%si, 0.0%st
Mem: 32707608k total, 30350532k used, 2357076k free, 1531552k buffers
Swap: 4456440k total, 0k used, 4456440k free, 24577068k cached

it is a four core processor :
[root@nagiosxi libexec]# nproc
4

[tgriep]

Can you run the following commands and post the output?

Code: Select all

top -b -n 1 |head -n 20
ps -ef --cols=300

Then run this command so show the IO Wait stats for your systems drives. After it runs a few times, stop it and post the output.

Code: Select all

iostat -x 1

[tgriep]

I found this link you can look at for troubleshooting the I/O Wait issue you are having. Take a look at it and see if this helps you in finding the process that is causing the Wait.
http://bencane.com/2012/08/06/troublesh ... -in-linux/

[ericssonvietnam]

I found this link you can look at for troubleshooting the I/O Wait issue you are having. Take a look at it and see if this helps you in finding the process that is causing the Wait.
http://bencane.com/2012/08/06/troublesh ... -in-linux/

Below is the output for the command.
I still have the warning for high system load
[root@nagiosxi ~]# uptime
01:54:14 up 1 day, 12 min, 1 user, load average: 9.92, 10.06, 10.0

[root@nagiosxi ~]# top -b -n 1 |head -n 20
top - 01:53:11 up 1 day, 11 min, 1 user, load average: 10.00, 10.07, 10.05
Tasks: 302 total, 1 running, 301 sleeping, 0 stopped, 0 zombie
Cpu(s): 15.3%us, 3.4%sy, 0.0%ni, 48.2%id, 33.0%wa, 0.0%hi, 0.1%si, 0.0%st
Mem: 32707608k total, 31703140k used, 1004468k free, 1409140k buffers
Swap: 4456440k total, 35232k used, 4421208k free, 26025556k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
423 root 20 0 0 0 0 D 2.0 0.0 9:05.12 jbd2/sda5-8
6669 root 20 0 97864 10m 1624 D 2.0 0.0 0:20.60 sendmail
13989 nagios 20 0 315m 28m 6904 S 2.0 0.1 0:00.13 php
45143 nagios 20 0 30232 9484 1500 S 2.0 0.0 6:25.84 nagios
1 root 20 0 19356 1320 1124 S 0.0 0.0 0:00.93 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
3 root RT 0 0 0 0 S 0.0 0.0 0:01.17 migration/0
4 root 20 0 0 0 0 S 0.0 0.0 0:09.58 ksoftirqd/0
5 root RT 0 0 0 0 S 0.0 0.0 0:00.00 stopper/0
6 root RT 0 0 0 0 S 0.0 0.0 0:00.10 watchdog/0
7 root RT 0 0 0 0 S 0.0 0.0 0:02.10 migration/1
8 root RT 0 0 0 0 S 0.0 0.0 0:00.00 stopper/1
9 root 20 0 0 0 0 S 0.0 0.0 0:07.86 ksoftirqd/1
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
attaching the output for iostat and ps command.Let me know how we can proceed.

[avandemore]

Can you install and run iotop? Have you installed a ramdisk? https://assets.nagios.com/downloads/nag ... giosXI.pdf

[ericssonvietnam]

Can you install and run iotop? Have you installed a ramdisk? https://assets.nagios.com/downloads/nag ... giosXI.pdf

As i run iotop below are the common process which are present in the output.
Can we check what is the data is being sent using the sendmail.
Can we stop sendmail if we are not using the mail service on the server.
Total DISK READ: 0.00 B/s | Total DISK WRITE: 1928.66 K/s
TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND
423 be/3 root 0.00 B/s 251.22 K/s 0.00 % 85.75 % [jbd2/sda5-8]
6669 be/4 root 0.00 B/s 15.46 K/s 0.00 % 9.85 % sendmail: ./v51IirTM002216 from queue
5280 be/4 root 0.00 B/s 15.46 K/s 0.00 % 8.16 % sendmail: ./v5263kHH001358 from queue
14509 be/4 root 0.00 B/s 15.46 K/s 0.00 % 8.00 % sendmail: ./v4UGH1Ck038522 from queue
7563 be/4 root 0.00 B/s 11.59 K/s 0.00 % 7.82 % sendmail: ./v51Iir3T002216 from queue
1398 be/3 root 0.00 B/s 38.65 K/s 0.00 % 7.79 % auditd
5634 be/4 root 0.00 B/s 15.46 K/s 0.00 % 7.25 % sendmail: ./v5251DoW002216 from queue
23657 be/4 root 0.00 B/s 11.59 K/s 0.00 % 6.35 % sendmail: ./v4VCs1G0012175 from queue
5982 be/4 root 0.00 B/s 11.59 K/s 0.00 % 6.25 % sendmail: ./v522RUng002216 from queue
47843 be/4 nagios 0.00 B/s 7.73 K/s 0.00 % 4.38 % sendmail: ./v52JQ1kp047843 [127.0.0.1]: client MAIL
47853 be/4 nagios 0.00 B/s 7.73 K/s 0.00 % 3.84 % sendmail: ./v52JQ1B3047853 [127.0.0.1]: client MAIL
1433 be/4 root 0.00 B/s 23.19 K/s 0.00 % 0.00 % rsyslogd -i /var/run/syslogd.pid -c 5
25957 be/4 apache 0.00 B/s 0.00 B/s 0.00 % 0.00 % httpd
2432 be/4 apache 0.00 B/s 3.86 K/s 0.00 % 0.00 % httpd
11298 be/4 root 0.00 B/s 3.86 K/s 0.00 % 0.00 % java -Djava.util.logging.config.file=/opt/apache-tomc~.0.65/temp org.apache.catalina.startup.Bootstrap start
2421 be/4 apache 0.00 B/s 3.86 K/s 0.00 % 0.00 % httpd
2439 be/4 mysql 0.00 B/s 0.00 B/s 0.00 % 0.00 % mysqld --basedir=/usr --datadir=/var/lib/mysql --user~n/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock
2494 be/4 nagios 0.00 B/s 3.86 K/s 0.00 % 0.00 % npcd -d -f /usr/local/nagios/etc/pnp/npcd.cfg
2617 be/4 mysql 0.00 B/s 0.00 B/s 0.00 % 0.00 % mysqld --basedir=/usr --datadir=/var/lib/mysql --user~n/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock
25896 be/4 apache 0.00 B/s 3.86 K/s 0.00 % 0.00 % httpd
25939 be/4 mysql 0.00 B/s 0.00 B/s 0.00 % 0.00 % mysqld --basedir=/usr --datadir=/var/lib/mysql --user~n/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock
26093 be/4 mysql 0.00 B/s 0.00 B/s 0.00 % 0.00 % mysqld --basedir=/usr --datadir=/var/lib/mysql --user~n/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock
48942 be/4 mysql 0.00 B/s 3.86 K/s 0.00 % 0.00 % mysqld --basedir=/usr --datadir=/var/lib/mysql --user~n/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock
48972 be/4 mysql 0.00 B/s 3.86 K/s 0.00 % 0.00 % mysqld --basedir=/usr --datadir=/var/lib/mysql --user~n/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock
1 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % init
2 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [kthreadd]
3 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration/0]
4 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [ksoftirqd/0]
5 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [stopper/0]
6 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [watchdog/0]

Can you suggest on the above point.