apparent memory issue doing upgrade to 2012R1.8

This support forum board is for support questions relating to Nagios XI, our flagship commercial network monitoring solution.
Locked
marquettegroup
Posts: 10
Joined: Mon Oct 22, 2012 9:56 am

apparent memory issue doing upgrade to 2012R1.8

Post by marquettegroup »

Linux Distribution and version? Centos 6.3
32 or 64bit? 64bit
VMware Image or Manual Install of XI? vmware image from nagios site
Are there specials configurations on your system: none

When I fire off backup_xi.sh the system seems to hang and no files/folders are written to backup folder. If I connect to server I see this on screen until i kill the ssh connection doing the upgrade:

Image
Top
sreinhardt
-fno-stack-protector
Posts: 4366
Joined: Mon Nov 19, 2012 12:10 pm

Re: apparent memory issue doing upgrade to 2012R1.8

Post by sreinhardt »

I will test this and let you know if I have the same issues. Did you perform anything other than attempting to backup?

Edit: I could not replicate this issue, it worked as expected for me.
Nagios-Plugins maintainer exclusively, unless you have other C language bugs with open-source nagios projects, then I am happy to help! Please pm or use other communication to alert me to issues as I no longer track the forum.
Top
marquettegroup
Posts: 10
Joined: Mon Oct 22, 2012 9:56 am

Re: apparent memory issue doing upgrade to 2012R1.8

Post by marquettegroup »

In trying to troubleshoot I have tested the db's(based on nagios article). I have done previous upgrades on this box.

I also have another nagios installation for sister company that worked fine.
Top
abrist
Red Shirt
Posts: 8334
Joined: Thu Nov 15, 2012 1:20 pm

Re: apparent memory issue doing upgrade to 2012R1.8

Post by abrist »

Lets check a couple of metrics on the Nagios XI system. Post the output of the following in code wraps:

Code: Select all

free -m
df -h
ps -aef
uname -a
What version of XI were you upgrading from?
Former Nagios employee
"It is turtles. All. The. Way. Down. . . .and maybe an elephant or two."
VI VI VI - The editor of the Beast!
Come to the Dark Side.
Top
marquettegroup
Posts: 10
Joined: Mon Oct 22, 2012 9:56 am

Re: apparent memory issue doing upgrade to 2012R1.8

Post by marquettegroup »

I'm upgrading from 2012R1.7.

here are requested info:

Code: Select all

[root@nagiosxi ~]# free -m
             total       used       free     shared    buffers     cached
Mem:          5849       1049       4799          0        148        219
-/+ buffers/cache:        682       5167
Swap:          255         21        234

[root@nagiosxi ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                      9.5G  2.4G  6.7G  27% /
tmpfs                 2.9G     0  2.9G   0% /dev/shm
/dev/sda1              97M   28M   65M  30% /boot

[root@nagiosxi ~]# uname -a
Linux nagiosxi 2.6.32-279.5.1.el6.x86_64 #1 SMP Tue Aug 14 23:54:45 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

Code: Select all

[root@nagiosxi ~]# ps -aef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 May08 ?        00:00:03 /sbin/init
root         2     0  0 May08 ?        00:00:00 [kthreadd]
root         3     2  0 May08 ?        00:00:00 [migration/0]
root         4     2  0 May08 ?        00:00:06 [ksoftirqd/0]
root         5     2  0 May08 ?        00:00:00 [migration/0]
root         6     2  0 May08 ?        00:00:21 [watchdog/0]
root         7     2  0 May08 ?        00:01:51 [events/0]
root         8     2  0 May08 ?        00:00:00 [cgroup]
root         9     2  0 May08 ?        00:00:13 [khelper]
root        10     2  0 May08 ?        00:00:00 [netns]
root        11     2  0 May08 ?        00:00:00 [async/mgr]
root        12     2  0 May08 ?        00:00:00 [pm]
root        13     2  0 May08 ?        00:00:07 [sync_supers]
root        14     2  0 May08 ?        00:00:06 [bdi-default]
root        15     2  0 May08 ?        00:00:00 [kintegrityd/0]
root        16     2  1 May08 ?        03:45:41 [kblockd/0]
root        17     2  0 May08 ?        00:00:00 [kacpid]
root        18     2  0 May08 ?        00:00:00 [kacpi_notify]
root        19     2  0 May08 ?        00:00:00 [kacpi_hotplug]
root        20     2  0 May08 ?        00:00:00 [ata/0]
root        21     2  0 May08 ?        00:00:00 [ata_aux]
root        22     2  0 May08 ?        00:00:00 [ksuspend_usbd]
root        23     2  0 May08 ?        00:00:00 [khubd]
root        24     2  0 May08 ?        00:00:00 [kseriod]
root        25     2  0 May08 ?        00:00:00 [md/0]
root        26     2  0 May08 ?        00:00:00 [md_misc/0]
root        27     2  0 May08 ?        00:00:00 [khungtaskd]
root        28     2  0 May08 ?        00:00:15 [kswapd0]
root        29     2  0 May08 ?        00:00:00 [ksmd]
root        30     2  0 May08 ?        00:02:47 [khugepaged]
root        31     2  0 May08 ?        00:00:00 [aio/0]
root        32     2  0 May08 ?        00:00:00 [crypto/0]
root        37     2  0 May08 ?        00:00:00 [kthrotld/0]
root        38     2  0 May08 ?        00:00:00 [pciehpd]
root        40     2  0 May08 ?        00:00:00 [kpsmoused]
root        41     2  0 May08 ?        00:00:00 [usbhid_resumer]
root        72     2  0 May08 ?        00:00:00 [kstriped]
root       277     2  0 May08 ?        00:00:00 [scsi_eh_0]
root       278     2  0 May08 ?        00:00:00 [scsi_eh_1]
root       285     2  0 May08 ?        00:00:35 [mpt_poll_0]
root       286     2  0 May08 ?        00:00:00 [mpt/0]
root       287     2  0 May08 ?        00:00:00 [scsi_eh_2]
root       356     2  0 May08 ?        00:00:00 [kdmflush]
root       358     2  0 May08 ?        00:00:00 [kdmflush]
root       378     2  0 May08 ?        00:49:45 [jbd2/dm-1-8]
root       379     2  0 May08 ?        00:00:00 [ext4-dio-unwrit]
root       457     1  0 May08 ?        00:00:00 /sbin/udevd -d
root       643     2  0 May08 ?        00:00:15 [vmmemctl]
root       771     2  0 May08 ?        02:05:09 [flush-253:1]
root       777   457  0 May08 ?        00:00:00 /sbin/udevd -d
root       880     2  0 May08 ?        00:00:07 [kauditd]
root      1083     1  0 May08 ?        00:08:37 auditd
root      1099     1  0 May08 ?        00:00:15 /sbin/rsyslogd -i /var/run/syslogd.pid -c 5
dbus      1215     1  0 May08 ?        00:00:00 dbus-daemon --system
root      1239     1  0 May08 ?        00:00:00 /usr/sbin/acpid
68        1248     1  0 May08 ?        00:00:08 hald
root      1249  1248  0 May08 ?        00:00:00 hald-runner
root      1277  1249  0 May08 ?        00:00:00 hald-addon-input: Listening on /dev/input/event2 /dev/input/event0
68        1287  1249  0 May08 ?        00:00:00 hald-addon-acpi: listening on acpid socket /var/run/acpid.socket
root      1307     1  0 May08 ?        00:00:04 /usr/sbin/sshd
root      1318     1  0 May08 ?        00:00:00 xinetd -stayalive -pidfile /var/run/xinetd.pid
postgres  1491     1  0 May08 ?        00:03:55 /usr/bin/postmaster -p 5432 -D /var/lib/pgsql/data
postgres  1555  1491  0 May08 ?        00:00:48 postgres: logger process
postgres  1596  1491  0 May08 ?        00:07:49 postgres: writer process
postgres  1597  1491  0 May08 ?        00:03:20 postgres: wal writer process
root      1598     1  0 May08 ?        00:00:04 /usr/libexec/postfix/master
postgres  1600  1491  0 May08 ?        00:01:29 postgres: autovacuum launcher process
postgres  1601  1491  0 May08 ?        00:01:53 postgres: stats collector process
postfix   1619  1598  0 May08 ?        00:00:01 qmgr -l -t fifo -u
root      1641     1  0 May08 ?        00:00:00 /usr/sbin/abrtd
root      1652     1  0 May08 ?        00:00:01 abrt-dump-oops -d /var/spool/abrt -rwx /var/log/messages
root      1668     1  0 May08 ?        00:00:57 /usr/sbin/httpd
root      1676     1  0 May08 ?        00:01:39 crond
nagios    1687     1  0 May08 ?        00:01:56 /usr/local/nagios/bin/npcd -d -f /usr/local/nagios/etc/pnp/npcd.cfg
root      1706     1  0 May08 ?        00:00:00 /usr/sbin/atd
ajaxterm  1716     1  0 May08 ?        00:04:42 python /usr/share/ajaxterm/ajaxterm.py --daemon --port=8022 --uid=ajaxterm
nagios    1794     1  0 May08 ?        00:00:00 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
root      1804     1  0 May08 tty2     00:00:00 /sbin/mingetty /dev/tty2
root      1806     1  0 May08 tty3     00:00:00 /sbin/mingetty /dev/tty3
root      1813     1  0 May08 tty4     00:00:00 /sbin/mingetty /dev/tty4
root      1820     1  0 May08 tty5     00:00:00 /sbin/mingetty /dev/tty5
root      1822     1  0 May08 tty6     00:00:00 /sbin/mingetty /dev/tty6
root      5556     1  0 04:55 ?        00:00:00 /usr/sbin/console-kit-daemon --no-daemon
root      5931     1  0 04:55 tty1     00:00:00 /sbin/mingetty /dev/tty1
root      5959  1307  0 04:55 ?        00:00:00 sshd: root@pts/0
root      6095  5959  0 04:56 pts/0    00:00:00 -bash
root      8592  1676  0 05:01 ?        00:00:00 CROND
root      8593  1676  0 05:01 ?        00:00:00 CROND
root      8594  1676  0 05:01 ?        00:00:00 CROND
root      8595  1676  0 05:01 ?        00:00:00 CROND
root      8596  1676  0 05:01 ?        00:00:00 CROND
nagios    8599  8593  0 05:01 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php > /usr/local/nagiosxi/var/feed
nagios    8600  8596  0 05:01 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php > /usr/local/nagiosxi/var/sysst
nagios    8603  8592  0 05:01 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/
nagios    8605  8594  0 05:01 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/even
nagios    8606  8595  0 05:01 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmd
nagios    8607  8600  1 05:01 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php
nagios    8609  8606  0 05:01 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php
nagios    8613  8599  0 05:01 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php
nagios    8614  8603  0 05:01 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php
nagios    8615  8605  1 05:01 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php
postgres  8640  1491  0 05:01 ?        00:00:00 postgres: nagiosxi nagiosxi 127.0.0.1(49595) idle
postgres  8643  1491  0 05:01 ?        00:00:00 postgres: nagiosxi nagiosxi 127.0.0.1(49598) idle
postgres  8644  1491  0 05:01 ?        00:00:00 postgres: nagiosxi nagiosxi 127.0.0.1(49599) idle
postgres  8667  1491  0 05:01 ?        00:00:00 postgres: nagiosxi nagiosxi 127.0.0.1(49602) idle
postgres  8741  1491  0 05:01 ?        00:00:00 postgres: nagiosxi nagiosxi 127.0.0.1(49606) idle
root      8821  6095  0 05:01 pts/0    00:00:00 ps -aef
nagios   16700  1794  0 May14 ?        00:00:30 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
nagios   16701 16700  0 May14 ?        00:02:48 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
nagios   16704     1  0 May14 ?        00:23:31 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
root     20736     1  0 May15 ?        00:00:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock --pid-
mysql    20828 20736  0 May15 ?        00:12:16 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --log-error=/var/log/m
apache   24769  1668  0 May15 ?        00:09:32 /usr/sbin/httpd
apache   24782  1668  0 May15 ?        00:10:25 /usr/sbin/httpd
apache   24811  1668  0 May15 ?        00:09:33 /usr/sbin/httpd
apache   24816  1668  0 May15 ?        00:09:35 /usr/sbin/httpd
apache   24817  1668  0 May15 ?        00:09:42 /usr/sbin/httpd
apache   24819  1668  0 May15 ?        00:10:27 /usr/sbin/httpd
apache   24822  1668  0 May15 ?        00:10:24 /usr/sbin/httpd
apache   24823  1668  0 May15 ?        00:09:41 /usr/sbin/httpd
apache   24883  1668  0 May15 ?        00:09:39 /usr/sbin/httpd
apache   24898  1668  0 May15 ?        00:09:31 /usr/sbin/httpd
apache   24916  1668  0 May15 ?        00:09:28 /usr/sbin/httpd
apache   24917  1668  0 May15 ?        00:09:34 /usr/sbin/httpd
apache   24918  1668  0 May15 ?        00:09:47 /usr/sbin/httpd
apache   24919  1668  0 May15 ?        00:09:24 /usr/sbin/httpd
postgres 24983  1491  0 May15 ?        00:00:12 postgres: nagiosxi nagiosxi 127.0.0.1(33702) idle
postgres 24989  1491  0 May15 ?        00:00:12 postgres: nagiosxi nagiosxi 127.0.0.1(33703) idle
postgres 25005  1491  0 May15 ?        00:00:12 postgres: nagiosxi nagiosxi 127.0.0.1(33704) idle
postgres 25027  1491  0 May15 ?        00:00:12 postgres: nagiosxi nagiosxi 127.0.0.1(33708) idle
postgres 25097  1491  0 May15 ?        00:00:12 postgres: nagiosxi nagiosxi 127.0.0.1(33717) idle
postgres 25115  1491  0 May15 ?        00:00:12 postgres: nagiosxi nagiosxi 127.0.0.1(33721) idle
postgres 25119  1491  0 May15 ?        00:00:12 postgres: nagiosxi nagiosxi 127.0.0.1(33722) idle
postgres 25157  1491  0 May15 ?        00:00:12 postgres: nagiosxi nagiosxi 127.0.0.1(33728) idle
postgres 25290  1491  0 May15 ?        00:00:12 postgres: nagiosxi nagiosxi 127.0.0.1(33748) idle
postgres 25609  1491  0 May15 ?        00:00:12 postgres: nagiosxi nagiosxi 127.0.0.1(33812) idle
postgres 25685  1491  0 May15 ?        00:00:12 postgres: nagiosxi nagiosxi 127.0.0.1(33816) idle
postgres 25699  1491  0 May15 ?        00:00:12 postgres: nagiosxi nagiosxi 127.0.0.1(33819) idle
postgres 25707  1491  0 May15 ?        00:00:12 postgres: nagiosxi nagiosxi 127.0.0.1(33821) idle
postgres 25749  1491  0 May15 ?        00:00:12 postgres: nagiosxi nagiosxi 127.0.0.1(33823) idle
postfix  26622  1598  0 04:34 ?        00:00:00 pickup -l -t fifo -u
[root@nagiosxi ~]# uname -a
Linux nagiosxi 2.6.32-279.5.1.el6.x86_64 #1 SMP Tue Aug 14 23:54:45 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
Last edited by sreinhardt on Fri May 17, 2013 11:10 am, edited 1 time in total.
Reason: Please use [code][/code] tags for large amounts of text. saves scoll wheels!
Top
abrist
Red Shirt
Posts: 8334
Joined: Thu Nov 15, 2012 1:20 pm

Re: apparent memory issue doing upgrade to 2012R1.8

Post by abrist »

Could you run those same metrics again *during* the upgrade? Your system specs seem fine, plenty of memory.
Former Nagios employee
"It is turtles. All. The. Way. Down. . . .and maybe an elephant or two."
VI VI VI - The editor of the Beast!
Come to the Dark Side.
Top
User avatar
lmiltchev
Bugs find me
Posts: 13589
Joined: Mon May 23, 2011 12:15 pm

Re: apparent memory issue doing upgrade to 2012R1.8

Post by lmiltchev »

Do you see anything interesting in the log files?

Code: Select all

tail /var/log/messages
tail /var/log/httpd/error_log
tail /usr/local/nagios/var/nagios.log
Be sure to check out our Knowledgebase for helpful articles and solutions!
Top
marquettegroup
Posts: 10
Joined: Mon Oct 22, 2012 9:56 am

Re: apparent memory issue doing upgrade to 2012R1.8

Post by marquettegroup »

abrist wrote:Could you run those same metrics again *during* the upgrade? Your system specs seem fine, plenty of memory.
[root@nagiosxi ~]# free -m
total used free shared buffers cached
Mem: 5849 1824 4025 0 159 301
-/+ buffers/cache: 1362 4486
Swap: 255 16 239
[root@nagiosxi ~]#
[root@nagiosxi ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
9.5G 2.4G 6.7G 26% /
tmpfs 2.9G 0 2.9G 0% /dev/shm
/dev/sda1 97M 28M 65M 30% /boot

Code: Select all


root     14988 14984  0 05:11 pts/0    00:00:00 -bash
root     14989 14988  0 05:11 pts/0    00:00:00 -bash
root     14990 14989  0 05:11 pts/0    00:00:00 -bash
root     14991 14990  0 05:11 pts/0    00:00:00 -bash
root     14992 14991  0 05:11 pts/0    00:00:00 -bash
root     14993 14992  0 05:11 pts/0    00:00:00 -bash
root     14994 14993  0 05:11 pts/0    00:00:00 -bash
root     14995 14994  0 05:11 pts/0    00:00:00 -bash
root     14996 14995  0 05:11 pts/0    00:00:00 -bash
root     14997 14996  0 05:11 pts/0    00:00:00 -bash
root     14998 14997  0 05:11 pts/0    00:00:00 -bash
root     14999 14998  0 05:11 pts/0    00:00:00 -bash
root     15000 14999  0 05:11 pts/0    00:00:00 -bash
root     15001 15000  0 05:11 pts/0    00:00:00 -bash
root     15002 15001  0 05:11 pts/0    00:00:00 -bash
root     15003 15002  0 05:11 pts/0    00:00:00 -bash
root     15004 15003  0 05:11 pts/0    00:00:00 -bash
root     15005 15004  0 05:11 pts/0    00:00:00 -bash
root     15006 15005  0 05:11 pts/0    00:00:00 -bash
root     15007 15006  0 05:11 pts/0    00:00:00 -bash
root     15008 15007  0 05:11 pts/0    00:00:00 -bash
root     15009 15008  0 05:11 pts/0    00:00:00 -bash
root     15010 15009  0 05:11 pts/0    00:00:00 -bash
root     15011 15010  0 05:11 pts/0    00:00:00 -bash
root     15012 15011  0 05:11 pts/0    00:00:00 -bash
root     15013 15012  0 05:11 pts/0    00:00:00 -bash
root     15014 15013  0 05:11 pts/0    00:00:00 -bash
root     15015 15014  0 05:11 pts/0    00:00:00 -bash
root     15016 15015  0 05:11 pts/0    00:00:00 -bash
root     15017 15016  0 05:11 pts/0    00:00:00 -bash
root     15018 15017  0 05:11 pts/0    00:00:00 -bash
root     15019 15018  0 05:11 pts/0    00:00:00 -bash
root     15020 15019  0 05:11 pts/0    00:00:00 -bash
root     15021 15020  0 05:11 pts/0    00:00:00 -bash
root     15022 15021  0 05:11 pts/0    00:00:00 -bash
root     15023 15022  0 05:11 pts/0    00:00:00 -bash
root     15025 15023  0 05:11 pts/0    00:00:00 -bash
root     15028 15025  0 05:11 pts/0    00:00:00 -bash
root     15029 15028  0 05:11 pts/0    00:00:00 -bash
root     15030 15029  0 05:11 pts/0    00:00:00 -bash
root     15031 15030  0 05:11 pts/0    00:00:00 -bash
root     15032 15031  0 05:11 pts/0    00:00:00 -bash
root     15033 15032  0 05:11 pts/0    00:00:00 -bash
root     15034 15033  0 05:11 pts/0    00:00:00 -bash
root     15035 15034  0 05:11 pts/0    00:00:00 -bash
root     15036 15035  0 05:11 pts/0    00:00:00 -bash
root     15037 15036  0 05:11 pts/0    00:00:00 -bash
root     15038 15037  0 05:11 pts/0    00:00:00 -bash
root     15039 15038  0 05:11 pts/0    00:00:00 -bash
root     15040 15039  0 05:11 pts/0    00:00:00 -bash
root     15041 15040  0 05:11 pts/0    00:00:00 -bash
root     15042 15041  0 05:11 pts/0    00:00:00 -bash
root     15043 15042  0 05:11 pts/0    00:00:00 -bash
root     15044 15043  0 05:11 pts/0    00:00:00 -bash
root     15045 15044  0 05:11 pts/0    00:00:00 -bash
root     15046 15045  0 05:11 pts/0    00:00:00 -bash
root     15047 15046  0 05:11 pts/0    00:00:00 -bash
root     15048 15047  0 05:11 pts/0    00:00:00 -bash
root     15049 15048  0 05:11 pts/0    00:00:00 -bash
root     15050 15049  0 05:11 pts/0    00:00:00 -bash
root     15051 15050  0 05:11 pts/0    00:00:00 -bash
root     15052 15051  0 05:11 pts/0    00:00:00 -bash
root     15055 15052  0 05:11 pts/0    00:00:00 -bash
root     15056 15055  0 05:11 pts/0    00:00:00 -bash
root     15057 15056  0 05:11 pts/0    00:00:00 -bash
root     15058 15057  0 05:11 pts/0    00:00:00 -bash
root     15059 15058  0 05:11 pts/0    00:00:00 -bash
root     15060 15059  0 05:11 pts/0    00:00:00 -bash
root     15061 15060  0 05:11 pts/0    00:00:00 -bash
root     15062 15061  0 05:11 pts/0    00:00:00 -bash
root     15063 15062  0 05:11 pts/0    00:00:00 -bash
root     15064 15063  0 05:11 pts/0    00:00:00 -bash
root     15065 15064  0 05:11 pts/0    00:00:00 -bash
root     15066 15065  0 05:11 pts/0    00:00:00 -bash
root     15067 15066  0 05:11 pts/0    00:00:00 -bash
root     15068 15067  0 05:11 pts/0    00:00:00 -bash
root     15069 15068  0 05:11 pts/0    00:00:00 -bash
root     15070 15069  0 05:11 pts/0    00:00:00 -bash
postgres 15071  1491  0 May19 ?        00:00:09 postgres: nagiosxi nagiosxi 127.0.0.1(36904) idle
root     15074 15070  0 05:11 pts/0    00:00:00 -bash
root     15075 15074  0 05:11 pts/0    00:00:00 -bash
root     15076 15075  0 05:11 pts/0    00:00:00 -bash
root     15077 15076  0 05:11 pts/0    00:00:00 -bash
root     15078 15077  0 05:11 pts/0    00:00:00 -bash
root     15079 15078  0 05:11 pts/0    00:00:00 -bash
root     15080 15079  0 05:11 pts/0    00:00:00 -bash
root     15081 15080  0 05:11 pts/0    00:00:00 -bash
root     15082 15081  0 05:11 pts/0    00:00:00 -bash
root     15083 15082  0 05:11 pts/0    00:00:00 -bash
root     15084 15083  0 05:11 pts/0    00:00:00 -bash
root     15085 15084  0 05:11 pts/0    00:00:00 -bash
root     15086 15085  0 05:11 pts/0    00:00:00 -bash
root     15088 15086  0 05:11 pts/0    00:00:00 -bash
root     15089 15088  0 05:11 pts/0    00:00:00 -bash
root     15090 15089  0 05:11 pts/0    00:00:00 -bash
root     15091 15090  0 05:11 pts/0    00:00:00 -bash
root     15094 15091  0 05:11 pts/0    00:00:00 -bash
root     15095 15094  0 05:11 pts/0    00:00:00 -bash
root     15096 15095  0 05:11 pts/0    00:00:00 -bash
root     15097 15096  0 05:11 pts/0    00:00:00 -bash
root     15098 15097  0 05:11 pts/0    00:00:00 -bash
root     15099 15098  0 05:11 pts/0    00:00:00 -bash
root     15100 15099  0 05:11 pts/0    00:00:00 -bash
root     15101 15100  0 05:11 pts/0    00:00:00 -bash
root     15107 15101  0 05:11 pts/0    00:00:00 -bash
root     15111 15107  0 05:11 pts/0    00:00:00 -bash
root     15112 15111  0 05:11 pts/0    00:00:00 -bash
root     15113 15112  0 05:11 pts/0    00:00:00 -bash
root     15114 15113  0 05:11 pts/0    00:00:00 -bash
root     15115 15114  0 05:12 pts/0    00:00:00 -bash
root     15116 15115  0 05:12 pts/0    00:00:00 -bash
root     15117 15116  0 05:12 pts/0    00:00:00 -bash
root     15118 15117  0 05:12 pts/0    00:00:00 -bash
root     15123 15118  0 05:12 pts/0    00:00:00 -bash
root     15126 15123  0 05:12 pts/0    00:00:00 -bash
root     15127 15126  0 05:12 pts/0    00:00:00 -bash
root     15128 15127  0 05:12 pts/0    00:00:00 -bash
root     15129 15128  0 05:12 pts/0    00:00:00 -bash
root     15132 15129  0 05:12 pts/0    00:00:00 -bash
root     15133 15132  0 05:12 pts/0    00:00:00 -bash
root     15134 15133  0 05:12 pts/0    00:00:00 -bash
root     15135 15134  0 05:12 pts/0    00:00:00 -bash
root     15136 15135  0 05:12 pts/0    00:00:00 -bash
root     15137 15136  0 05:12 pts/0    00:00:00 -bash
root     15138 15137  0 05:12 pts/0    00:00:00 -bash
root     15142  1676  0 05:12 ?        00:00:00 CROND
root     15143  1676  0 05:12 ?        00:00:00 CROND
root     15144  1676  0 05:12 ?        00:00:00 CROND
root     15145  1676  0 05:12 ?        00:00:00 CROND
root     15146  1676  0 05:12 ?        00:00:00 CROND
nagios   15151 15142  0 05:12 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php > /usr/local/nagiosxi/var/perfdataproc.log 2>&1
nagios   15152 15144  0 05:12 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php > /usr/local/nagiosxi/var/eventman.log 2>&1
nagios   15153 15143  0 05:12 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php > /usr/local/nagiosxi/var/feedproc.log 2>&1
nagios   15155 15145  0 05:12 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php > /usr/local/nagiosxi/var/cmdsubsys.log 2>&1
nagios   15156 15146  0 05:12 ?        00:00:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php > /usr/local/nagiosxi/var/sysstat.log 2>&1
nagios   15161 15151  2 05:12 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php
nagios   15162 15153  1 05:12 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php
nagios   15163 15152  2 05:12 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php
nagios   15164 15155  2 05:12 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php
nagios   15165 15156  2 05:12 ?        00:00:00 /usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php
root     15168 15138  0 05:12 pts/0    00:00:00 -bash
root     15172 15168  0 05:12 pts/0    00:00:00 -bash
root     15175 15172  0 05:12 pts/0    00:00:00 -bash
root     15177 15175  0 05:12 pts/0    00:00:00 -bash
root     15178 15177  0 05:12 pts/0    00:00:00 -bash
root     15179 15178  0 05:12 pts/0    00:00:00 -bash
root     15182 15179  0 05:12 pts/0    00:00:00 -bash
postgres 15197  1491  0 05:12 ?        00:00:00 postgres: nagiosxi nagiosxi 127.0.0.1(52070) idle
postgres 15199  1491  0 05:12 ?        00:00:00 postgres: nagiosxi nagiosxi 127.0.0.1(52071) idle
postgres 15200  1491  0 05:12 ?        00:00:00 postgres: nagiosxi nagiosxi 127.0.0.1(52072) idle
root     15210 15182  0 05:12 pts/0    00:00:00 -bash
root     15216 15210  0 05:12 pts/0    00:00:00 -bash
root     15225 15216  0 05:12 pts/0    00:00:00 -bash
postgres 15232  1491  0 05:12 ?        00:00:00 postgres: nagiosxi nagiosxi 127.0.0.1(52076) idle
root     15239 15225  0 05:12 pts/0    00:00:00 -bash
root     15242 15239  0 05:12 pts/0    00:00:00 -bash
root     15251 15242  0 05:12 pts/0    00:00:00 -bash
root     15266 15251  0 05:12 pts/0    00:00:00 -bash
root     15268 15266  0 05:12 pts/0    00:00:00 -bash
root     15273 15268  0 05:12 pts/0    00:00:00 -bash
root     15274 15273  0 05:12 pts/0    00:00:00 -bash
root     15275 15274  0 05:12 pts/0    00:00:00 -bash
root     15282 15275  0 05:12 pts/0    00:00:00 -bash
root     15285 15282  0 05:12 pts/0    00:00:00 -bash
root     15287 15285  0 05:12 pts/0    00:00:00 -bash
root     15290 15287  0 05:12 pts/0    00:00:00 -bash
root     15291 15290  0 05:12 pts/0    00:00:00 -bash
root     15292 15291  0 05:12 pts/0    00:00:00 -bash
root     15293 15292  0 05:12 pts/0    00:00:00 -bash
root     15295 15293  0 05:12 pts/0    00:00:00 -bash
root     15302 15295  0 05:12 pts/0    00:00:00 -bash
root     15307 15302  0 05:12 pts/0    00:00:00 -bash
root     15310 15307  0 05:12 pts/0    00:00:00 -bash
root     15311 15310  0 05:12 pts/0    00:00:00 -bash
root     15312 15311  0 05:12 pts/0    00:00:00 -bash
root     15317 15312  0 05:12 pts/0    00:00:00 -bash
root     15325 15317  0 05:12 pts/0    00:00:00 -bash
root     15334 15325  0 05:12 pts/0    00:00:00 -bash
root     15335 15334  0 05:12 pts/0    00:00:00 -bash
root     15338 15335  0 05:12 pts/0    00:00:00 -bash
root     15343 15338  0 05:12 pts/0    00:00:00 -bash
root     15346 15343  0 05:12 pts/0    00:00:00 -bash
root     15347 15346  0 05:12 pts/0    00:00:00 -bash
root     15348 15347  0 05:12 pts/0    00:00:00 -bash
root     15349 15348  0 05:12 pts/0    00:00:00 -bash
root     15350 15349  0 05:12 pts/0    00:00:00 -bash
root     15351 15350  0 05:12 pts/0    00:00:00 -bash
root     15352 15351  0 05:12 pts/0    00:00:00 -bash
root     15357 15352  0 05:12 pts/0    00:00:00 -bash
root     15360 15357  0 05:12 pts/0    00:00:00 -bash
root     15363 15360  0 05:12 pts/0    00:00:00 -bash
root     15364 15363  0 05:12 pts/0    00:00:00 -bash
root     15365 15364  0 05:12 pts/0    00:00:00 -bash
root     15366 15365  0 05:12 pts/0    00:00:00 -bash
root     15367 15366  0 05:12 pts/0    00:00:00 -bash
root     15368 15367  0 05:12 pts/0    00:00:00 -bash
root     15369 15368  0 05:12 pts/0    00:00:00 -bash
root     15376 15369  0 05:12 pts/0    00:00:00 -bash
root     15379 15376  0 05:12 pts/0    00:00:00 -bash
root     15382 15379  0 05:12 pts/0    00:00:00 -bash
root     15383 15382  0 05:12 pts/0    00:00:00 -bash
root     15384 15383  0 05:12 pts/0    00:00:00 -bash
root     15385 15384  0 05:12 pts/0    00:00:00 -bash
root     15386 15385  0 05:12 pts/0    00:00:00 -bash
root     15387 15386  0 05:12 pts/0    00:00:00 -bash
root     15388 15387  0 05:12 pts/0    00:00:00 -bash
root     15393 15388  0 05:12 pts/0    00:00:00 -bash
root     15397 15393  0 05:12 pts/0    00:00:00 -bash
root     15399 15397  0 05:12 pts/0    00:00:00 -bash
root     15400 15399  1 05:12 pts/0    00:00:00 -bash
root     15401 15400  0 05:12 pts/0    00:00:00 -bash
root     15402 15401  1 05:12 pts/0    00:00:00 -bash
root     15403 15402  0 05:12 pts/0    00:00:00 -bash
root     15404 15403  1 05:12 pts/0    00:00:00 -bash
root     15405 15404  1 05:12 pts/0    00:00:00 -bash
root     15411 15405  1 05:12 pts/0    00:00:00 -bash
root     15413 15411  1 05:12 pts/0    00:00:00 -bash
root     15414 15413  0 05:12 pts/0    00:00:00 -bash
root     15415 15414  1 05:12 pts/0    00:00:00 -bash
root     15418 15415  1 05:12 pts/0    00:00:00 -bash
root     15419 15418  0 05:12 pts/0    00:00:00 -bash
root     15420 15419  1 05:12 pts/0    00:00:00 -bash
root     15421 15420  0 05:12 pts/0    00:00:00 -bash
root     15422 15421  1 05:12 pts/0    00:00:00 -bash
root     15423 15422  1 05:12 pts/0    00:00:00 -bash
root     15424 15423  1 05:12 pts/0    00:00:00 -bash
root     15429 15424  1 05:12 pts/0    00:00:00 -bash
root     15432 15429  1 05:12 pts/0    00:00:00 -bash
nagios   15436 16704  0 05:12 ?        00:00:00 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios   15439 15436  0 05:12 ?        00:00:00 /usr/local/nagios/libexec/check_http -H www.allconnectrewards.com
nagios   15443 16704  0 05:12 ?        00:00:00 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios   15444 16704  0 05:12 ?        00:00:00 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios   15445 15443  0 05:12 ?        00:00:00 /usr/local/nagios/libexec/check_http -H mycitizensrewards.com
nagios   15446 15444  0 05:12 ?        00:00:00 /usr/local/nagios/libexec/check_http -H routewarriorsrewards.com
nagios   15454 16704  0 05:12 ?        00:00:00 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios   15456 15454  0 05:12 ?        00:00:00 /usr/local/nagios/libexec/check_http -H smas.ovationv3.com
root     15460 15432  1 05:12 pts/0    00:00:00 -bash
nagios   15461 16704  0 05:12 ?        00:00:00 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
nagios   15463 15461  0 05:12 ?        00:00:00 /usr/local/nagios/libexec/check_http -H southwire.ovation-online.net
root     15464 15460  1 05:12 pts/0    00:00:00 -bash
root     15465 15464  1 05:12 pts/0    00:00:00 -bash
root     15466 15465  3 05:12 pts/0    00:00:00 -bash
root     15469 15466  3 05:12 pts/0    00:00:00 -bash
root     15470 15469  3 05:12 pts/0    00:00:00 -bash
root     15476 15470  3 05:12 pts/0    00:00:00 -bash
root     15478 15476  3 05:12 pts/0    00:00:00 -bash
root     15479 15478  3 05:12 pts/0    00:00:00 -bash
root     15480 12070 35 05:12 pts/1    00:00:00 ps -aef
root     15481 15479  3 05:12 pts/0    00:00:00 -bash
root     15483 15481  3 05:12 pts/0    00:00:00 -bash
root     15485 15483  0 05:12 pts/0    00:00:00 -bash
root     15486 15485  0 05:12 pts/0    00:00:00 -bash
root     15487 15486  0 05:12 pts/0    00:00:00 -bash
apache   16355  1668  0 May19 ?        00:06:58 /usr/sbin/httpd
postgres 16619  1491  0 May19 ?        00:00:08 postgres: nagiosxi nagiosxi 127.0.0.1(37127) idle
nagios   16700  1794  0 May14 ?        00:01:07 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
nagios   16701 16700  0 May14 ?        00:06:23 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg
nagios   16704     1  0 May14 ?        00:51:08 /usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
apache   19076  1668  0 May19 ?        00:07:05 /usr/sbin/httpd
postgres 19294  1491  0 May19 ?        00:00:09 postgres: nagiosxi nagiosxi 127.0.0.1(37522) idle
root     20736     1  0 May15 ?        00:00:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock --pid-file=/var/run/mysqld/mysqld.pid --basedir=/usr --user=mysql
mysql    20828 20736  0 May15 ?        00:32:34 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --log-error=/var/log/mysqld.log --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock
postfix  24621  1598  0 04:08 ?        00:00:00 pickup -l -t fifo -u
apache   27272  1668  0 May19 ?        00:06:36 /usr/sbin/httpd
Top
marquettegroup
Posts: 10
Joined: Mon Oct 22, 2012 9:56 am

Re: apparent memory issue doing upgrade to 2012R1.8

Post by marquettegroup »

lmiltchev wrote:Do you see anything interesting in the log files?

Code: Select all

[root@nagiosxi ~]# tail /var/log/messages
May 20 05:16:25 nagiosxi kernel: [13680]     0 13680    27604      548   0       0             0 bash
May 20 05:16:25 nagiosxi kernel: [13681]     0 13681    27604      549   0       0             0 bash
May 20 05:16:25 nagiosxi kernel: [13682]     0 13682    27604      550   0       0             0 bash
May 20 05:16:25 nagiosxi kernel: [13683]     0 13683    27604      550   0       0             0 bash
May 20 05:16:25 nagiosxi kernel: [13684]     0 13684    27604      551   0       0             0 bash
May 20 05:16:25 nagiosxi kernel: [13685]     0 13685    27604      552   0       0             0 bash
May 20 05:16:25 nagiosxi kernel: [13686]     0 13686    27604      553   0       0             0 bash
May 20 05:16:25 nagiosxi kernel: [13687]     0 13687    27604      553   0       0             0 bash
May 20 05:16:25 nagiosxi kernel: [13688]     0 13688    27637      554   0       0             0 bash
May [root@nagiosxi ~]#

Code: Select all

[root@nagiosxi ~]# tail /var/log/httpd/error_log
[Mon May 20 05:13:33 2013] [error] [client 192.168.52.9] PHP Warning:  mysql_pconnect(): Link to server lost, unable to reconnect in /usr/local/nagiosxi/html/db/adodb/drivers/adodb-mysql.inc.php on line 383, referer: http://nagiosxi.usm.local/nagiosxi/includes/components/xicore/status.php?show=services&hoststatustypes=2&servicestatustypes=28&serviceattr=10
[Mon May 20 05:13:33 2013] [error] [client 192.168.52.9] PHP Warning:  mysql_pconnect(): Link to server lost, unable to reconnect in /usr/local/nagiosxi/html/db/adodb/drivers/adodb-mysql.inc.php on line 383, referer: http://nagiosxi.usm.local/nagiosxi/includes/components/xicore/status.php?show=services&hoststatustypes=2&servicestatustypes=28&serviceattr=10
[Mon May 20 05:13:33 2013] [error] [client 192.168.52.9] PHP Warning:  mysql_pconnect(): Link to server lost, unable to reconnect in /usr/local/nagiosxi/html/db/adodb/drivers/adodb-mysql.inc.php on line 383, referer: http://nagiosxi.usm.local/nagiosxi/views/index.php?
[Mon May 20 05:13:33 2013] [error] [client 192.168.52.9] PHP Warning:  mysql_pconnect(): Link to server lost, unable to reconnect in /usr/local/nagiosxi/html/db/adodb/drivers/adodb-mysql.inc.php on line 383, referer: http://nagiosxi.usm.local/nagiosxi/views/index.php?
[Mon May 20 05:13:34 2013] [error] [client 192.168.52.9] PHP Warning:  mysql_pconnect(): Link to server lost, unable to reconnect in /usr/local/nagiosxi/html/db/adodb/drivers/adodb-mysql.inc.php on line 383, referer: http://nagiosxi.usm.local/nagiosxi/includes/components/xicore/status.php?show=services&hoststatustypes=2&servicestatustypes=28&serviceattr=10
[Mon May 20 05:13:34 2013] [error] [client 192.168.52.9] PHP Warning:  mysql_pconnect(): Link to server lost, unable to reconnect in /usr/local/nagiosxi/html/db/adodb/drivers/adodb-mysql.inc.php on line 383, referer: http://nagiosxi.usm.local/nagiosxi/includes/components/xicore/status.php?show=services&hoststatustypes=2&servicestatustypes=28&serviceattr=10
[Mon May 20 05:13:34 2013] [error] [client 192.168.52.9] PHP Warning:  mysql_pconnect(): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2) in /usr/local/nagiosxi/html/db/adodb/drivers/adodb-mysql.inc.php on line 383, referer: http://nagiosxi.usm.local/nagiosxi/includes/components/xicore/status.php?show=services&hoststatustypes=2&servicestatustypes=28&serviceattr=10
[Mon May 20 05:13:34 2013] [error] [client 192.168.52.9] PHP Warning:  mysql_pconnect(): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2) in /usr/local/nagiosxi/html/db/adodb/drivers/adodb-mysql.inc.php on line 383, referer: http://nagiosxi.usm.local/nagiosxi/includes/components/xicore/status.php?show=services&hoststatustypes=2&servicestatustypes=28&serviceattr=10
[Mon May 20 05:13:35 2013] [error] [client 192.168.52.9] PHP Warning:  mysql_pconnect(): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2) in /usr/local/nagiosxi/html/db/adodb/drivers/adodb-mysql.inc.php on line 383, referer: http://nagiosxi.usm.local/nagiosxi/views/index.php?
[Mon May 20 05:13:35 2013] [error] [client 192.168.52.9] PHP Warning:  mysql_pconnect(): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2) in /usr/local/nagiosxi/html/db/adodb/drivers/adodb-mysql.inc.php on line 383, referer: http://nagiosxi.usm.local/nagiosxi/views/index.php?
[root@nagiosxi ~]#

Code: Select all

[root@nagiosxi ~]# tail /usr/local/nagios/var/nagios.log
[1369041780] SERVICE ALERT: karastanimagination.com;Web Page Content;OK;SOFT;2;HTTP OK - HTTP/1.1 302 Moved Temporarily - 3.078 second response time
[1369041780] SERVICE ALERT: localhost;Current Load;CRITICAL;SOFT;2;CRITICAL - load average: 216.05, 253.88, 116.10
[1369041830] SERVICE ALERT: localhost;HTTP;CRITICAL;SOFT;3;Connection refused
[1369041840] SERVICE ALERT: localhost;Current Load;CRITICAL;SOFT;3;CRITICAL - load average: 79.42, 207.70, 108.84
[1369041890] SERVICE ALERT: localhost;HTTP;CRITICAL;HARD;4;Connection refused
[1369041890] SERVICE NOTIFICATION: nagiosadmin;localhost;HTTP;CRITICAL;xi_service_notification_handler;Connection refused
[1369041900] SERVICE ALERT: localhost;Current Load;CRITICAL;HARD;4;CRITICAL - load average: 29.17, 169.90, 102.03
[1369041900] SERVICE NOTIFICATION: nagiosadmin;localhost;Current Load;CRITICAL;xi_service_notification_handler;CRITICAL - load average: 29.17, 169.90, 102.03
[1369042030] SERVICE ALERT: smas.ovationv3.com;_login.aspx URL Content;CRITICAL;SOFT;1;CRITICAL - Socket timeout after 10 seconds
[1369042080] SERVICE ALERT: smas.ovationv3.com;_login.aspx URL Content;OK;SOFT;2;HTTP OK HTTP/1.1 200 OK - 1.089 second response time
[root@nagiosxi ~]#
Top
User avatar
lmiltchev
Bugs find me
Posts: 13589
Joined: Mon May 23, 2011 12:15 pm

Re: apparent memory issue doing upgrade to 2012R1.8

Post by lmiltchev »

Is mysql offloaded? Can you post the "/usr/local/nagios/etc/ndo2db.cfg"?
Be sure to check out our Knowledgebase for helpful articles and solutions!
Top
Locked

Return to “Nagios XI”