db repair failed

This support forum board is for support questions relating to Nagios XI, our flagship commercial network monitoring solution.
rtsupport
Posts: 188
Joined: Mon Jan 08, 2018 4:30 am

db repair failed

Post by rtsupport »

Hi Team,

after patching we rebooted our server after which we faced lots of issue.

- Notification are disabled but still naigos sending notification to the users
- permission for /usr/local/nagios/libexec changed as root:nagios for all plugins
- after restart trying to repair the db using /repair_databases.sh and it found multiple entry are corrupted which it trying to repair but due to space issue its failed to repair. and upon checking on dir below directory are consuming more space .

so now question is ?

can these files deleted ?
why these files are created so frequently and how to stop them?

5.0G mysql
7.0G pgsql

/var/lib/mysql/nagios

Code: Select all

-rw-rw---- 1 mysql mysql         61 Sep 10 18:20 db.opt
-rw-rw---- 1 mysql mysql       9100 Sep 10 18:20 nagios_acknowledgements.frm
-rw-rw---- 1 mysql mysql       3624 Jan 15 09:37 nagios_acknowledgements.MYD
-rw-rw---- 1 mysql mysql       3072 Jan 15 09:37 nagios_acknowledgements.MYI
-rw-rw---- 1 mysql mysql       8746 Sep 10 18:20 nagios_commands.frm
-rw-rw---- 1 mysql mysql      45156 Jan 15 09:37 nagios_commands.MYD
-rw-rw---- 1 mysql mysql      15360 Jan 15 09:37 nagios_commands.MYI
-rw-rw---- 1 mysql mysql       9340 Sep 10 18:20 nagios_commenthistory.frm
-rw-rw---- 1 mysql mysql   34163824 Jan 15 09:37 nagios_commenthistory.MYD
-rw-rw---- 1 mysql mysql    3337216 Jan 15 09:37 nagios_commenthistory.MYI
-rw-rw---- 1 mysql mysql       9220 Sep 10 18:20 nagios_comments.frm
-rw-rw---- 1 mysql mysql      29512 Jan 15 09:37 nagios_comments.MYD
-rw-rw---- 1 mysql mysql       8192 Jan 15 09:37 nagios_comments.MYI
-rw-rw---- 1 mysql mysql       8726 Sep 10 18:20 nagios_configfiles.frm
-rw-rw---- 1 mysql mysql         44 Jan 15 09:37 nagios_configfiles.MYD
-rw-rw---- 1 mysql mysql       5120 Jan 15 09:37 nagios_configfiles.MYI
-rw-rw---- 1 mysql mysql       8760 Sep 10 18:20 nagios_configfilevariables.frm
-rw-rw---- 1 mysql mysql       7120 Jan 15 09:37 nagios_configfilevariables.MYD
-rw-rw---- 1 mysql mysql       4096 Jan 15 09:37 nagios_configfilevariables.MYI
-rw-rw---- 1 mysql mysql       9262 Sep 10 18:20 nagios_conninfo.frm
-rw-rw---- 1 mysql mysql      48876 Jan 15 09:37 nagios_conninfo.MYD
-rw-rw---- 1 mysql mysql       8192 Jan 15 09:37 nagios_conninfo.MYI
-rw-rw---- 1 mysql mysql       8760 Sep 10 18:20 nagios_contact_addresses.frm
-rw-rw---- 1 mysql mysql          0 Jan 15 09:37 nagios_contact_addresses.MYD
-rw-rw---- 1 mysql mysql       1024 Jan 15 09:37 nagios_contact_addresses.MYI
-rw-rw---- 1 mysql mysql       8748 Sep 10 18:20 nagios_contactgroup_members.frm
-rw-rw---- 1 mysql mysql      13635 Jan 15 09:37 nagios_contactgroup_members.MYD
-rw-rw---- 1 mysql mysql      25600 Jan 15 09:37 nagios_contactgroup_members.MYI
-rw-rw---- 1 mysql mysql       8768 Sep 10 18:20 nagios_contactgroups.frm
-rw-rw---- 1 mysql mysql       9484 Jan 15 09:37 nagios_contactgroups.MYD
-rw-rw---- 1 mysql mysql      10240 Jan 15 09:37 nagios_contactgroups.MYI
-rw-rw---- 1 mysql mysql       8856 Sep 10 18:20 nagios_contact_notificationcommands.frm
-rw-rw---- 1 mysql mysql      32208 Jan 15 09:37 nagios_contact_notificationcommands.MYD
-rw-rw---- 1 mysql mysql      36864 Jan 15 09:37 nagios_contact_notificationcommands.MYI
-rw-rw---- 1 mysql mysql       9000 Sep 10 18:20 nagios_contactnotificationmethods.frm
-rw-rw---- 1 mysql mysql    4223584 Jan 15 09:37 nagios_contactnotificationmethods.MYD
-rw-rw---- 1 mysql mysql    4427776 Jan 15 09:37 nagios_contactnotificationmethods.MYI
-rw-rw---- 1 mysql mysql       8928 Sep 10 18:20 nagios_contactnotifications.frm
-rw-rw---- 1 mysql mysql    3784482 Jan 15 09:37 nagios_contactnotifications.MYD
-rw-rw---- 1 mysql mysql    5411840 Jan 15 09:37 nagios_contactnotifications.MYI
-rw-rw---- 1 mysql mysql      10026 Sep 10 18:20 nagios_contacts.frm
-rw-rw---- 1 mysql mysql      12092 Jan 15 09:37 nagios_contacts.MYD
-rw-rw---- 1 mysql mysql       7168 Jan 15 09:37 nagios_contacts.MYI
-rw-rw---- 1 mysql mysql       9240 Sep 10 18:20 nagios_contactstatus.frm
-rw-rw---- 1 mysql mysql       6222 Jan 15 09:37 nagios_contactstatus.MYD
-rw-rw---- 1 mysql mysql       7168 Jan 15 09:37 nagios_contactstatus.MYI
-rw-rw---- 1 mysql mysql       8844 Sep 10 18:20 nagios_customvariables.frm
-rw-rw---- 1 mysql mysql     259080 Jan 15 09:37 nagios_customvariables.MYD
-rw-rw---- 1 mysql mysql     233472 Jan 15 09:37 nagios_customvariables.MYI
-rw-rw---- 1 mysql mysql       8870 Sep 10 18:20 nagios_customvariablestatus.frm
-rw-rw---- 1 mysql mysql     314352 Jan 15 09:37 nagios_customvariablestatus.MYD
-rw-rw---- 1 mysql mysql     231424 Jan 15 09:37 nagios_customvariablestatus.MYI
-rw-rw---- 1 mysql mysql       8596 Sep 10 18:20 nagios_dbversion.frm
-rw-rw---- 1 mysql mysql         20 Jan 15 09:37 nagios_dbversion.MYD
-rw-rw---- 1 mysql mysql       1024 Jan 15 09:37 nagios_dbversion.MYI
-rw-rw---- 1 mysql mysql       9490 Sep 10 18:20 nagios_downtimehistory.frm
-rw-rw---- 1 mysql mysql     268328 Jan 15 09:37 nagios_downtimehistory.MYD
-rw-rw---- 1 mysql mysql      81920 Jan 15 09:37 nagios_downtimehistory.MYI
-rw-rw---- 1 mysql mysql       9380 Sep 10 18:20 nagios_eventhandlers.frm
-rw-rw---- 1 mysql mysql      71464 Jan 15 09:37 nagios_eventhandlers.MYD
-rw-rw---- 1 mysql mysql      12288 Jan 15 09:37 nagios_eventhandlers.MYI
-rw-rw---- 1 mysql mysql       8812 Sep 10 18:20 nagios_externalcommands.frm
-rw-rw---- 1 mysql mysql       9064 Jan 15 09:37 nagios_externalcommands.MYD
-rw-rw---- 1 mysql mysql       2048 Jan 15 09:37 nagios_externalcommands.MYI
-rw-rw---- 1 mysql mysql       9166 Sep 10 18:20 nagios_flappinghistory.frm
-rw-rw---- 1 mysql mysql   11107850 Jan 15 09:37 nagios_flappinghistory.MYD
-rw-rw---- 1 mysql mysql    1743872 Jan 15 09:37 nagios_flappinghistory.MYI
-rw-rw---- 1 mysql mysql       9660 Sep 10 18:20 nagios_hostchecks.frm
-rw-rw---- 1 mysql mysql          0 Jan 15 09:37 nagios_hostchecks.MYD
-rw-rw---- 1 mysql mysql       1024 Jan 15 09:37 nagios_hostchecks.MYI
-rw-rw---- 1 mysql mysql       8738 Sep 10 18:20 nagios_host_contactgroups.frm
-rw-rw---- 1 mysql mysql       1695 Jan 15 09:37 nagios_host_contactgroups.MYD
-rw-rw---- 1 mysql mysql       7168 Jan 15 09:37 nagios_host_contactgroups.MYI
-rw-rw---- 1 mysql mysql       8718 Sep 10 18:20 nagios_host_contacts.frm
-rw-rw---- 1 mysql mysql        105 Jan 15 09:37 nagios_host_contacts.MYD
-rw-rw---- 1 mysql mysql       3072 Jan 15 09:37 nagios_host_contacts.MYI
-rw-rw---- 1 mysql mysql       9108 Sep 10 18:20 nagios_hostdependencies.frm
-rw-rw---- 1 mysql mysql          0 Jan 15 09:37 nagios_hostdependencies.MYD
-rw-rw---- 1 mysql mysql       1024 Jan 15 09:37 nagios_hostdependencies.MYI
-rw-rw---- 1 mysql mysql       8778 Sep 10 18:20 nagios_hostescalation_contactgroups.frm
-rw-rw---- 1 mysql mysql          0 Jan 15 09:37 nagios_hostescalation_contactgroups.MYD
-rw-rw---- 1 mysql mysql       1024 Jan 15 09:37 nagios_hostescalation_contactgroups.MYI
-rw-rw---- 1 mysql mysql       8758 Sep 10 18:20 nagios_hostescalation_contacts.frm
-rw-rw---- 1 mysql mysql          0 Jan 15 09:37 nagios_hostescalation_contacts.MYD
-rw-rw---- 1 mysql mysql       1024 Jan 15 09:37 nagios_hostescalation_contacts.MYI
-rw-rw---- 1 mysql mysql       9148 Sep 10 18:20 nagios_hostescalations.frm
-rw-rw---- 1 mysql mysql          0 Jan 15 09:37 nagios_hostescalations.MYD
-rw-rw---- 1 mysql mysql       1024 Jan 15 09:37 nagios_hostescalations.MYI
-rw-rw---- 1 mysql mysql       8730 Sep 10 18:20 nagios_hostgroup_members.frm
-rw-rw---- 1 mysql mysql      41175 Jan 15 09:37 nagios_hostgroup_members.MYD
-rw-rw---- 1 mysql mysql      69632 Jan 15 09:37 nagios_hostgroup_members.MYI
-rw-rw---- 1 mysql mysql       8756 Sep 10 18:20 nagios_hostgroups.frm
-rw-rw---- 1 mysql mysql      17192 Jan 15 09:37 nagios_hostgroups.MYD
-rw-rw---- 1 mysql mysql      18432 Jan 15 09:37 nagios_hostgroups.MYI
-rw-rw---- 1 mysql mysql       8732 Sep 10 18:20 nagios_host_parenthosts.frm
-rw-rw---- 1 mysql mysql          0 Jan 15 09:37 nagios_host_parenthosts.MYD
-rw-rw---- 1 mysql mysql       1024 Jan 15 09:37 nagios_host_parenthosts.MYI
-rw-rw---- 1 mysql mysql      16028 Sep 10 18:20 nagios_hosts.frm
-rw-rw---- 1 mysql mysql     150728 Jan 15 09:37 nagios_hosts.MYD
-rw-rw---- 1 mysql mysql      32768 Jan 15 09:37 nagios_hosts.MYI
-rw-rw---- 1 mysql mysql      19464 Sep 10 18:20 nagios_hoststatus.frm
-rw-rw---- 1 mysql mysql     262920 Jan 15 09:37 nagios_hoststatus.MYD
-rw-rw---- 1 mysql mysql     172032 Jan 15 09:37 nagios_hoststatus.MYI
-rw-rw---- 1 mysql mysql       8684 Sep 10 18:20 nagios_instances.frm
-rw-rw---- 1 mysql mysql         20 Jan 15 09:37 nagios_instances.MYD
-rw-rw---- 1 mysql mysql       2048 Jan 15 09:37 nagios_instances.MYI
-rw-rw---- 1 mysql mysql       8972 Sep 10 18:20 nagios_logentries.frm
-rw-rw---- 1 mysql mysql 1356122380 Jan 11 23:40 nagios_logentries.MYD
-rw-rw---- 1 mysql mysql  440775680 Jan 15 09:38 nagios_logentries.MYI
-rw------- 1 root  root  1354833992 Jan 15 09:37 nagios_logentries.TMD
-rw-rw---- 1 mysql mysql       9168 Sep 10 18:20 nagios_notifications.frm
-rw-rw---- 1 mysql mysql  362768380 Jan 15 05:21 nagios_notifications.MYD
-rw-rw---- 1 mysql mysql  121378816 Jan 15 06:40 nagios_notifications.MYI
-rw-rw---- 1 mysql mysql       8766 Sep 10 18:20 nagios_objects.frm
-rw-rw---- 1 mysql mysql     456328 Jan 15 06:30 nagios_objects.MYD
-rw-rw---- 1 mysql mysql     484352 Jan 15 06:40 nagios_objects.MYI
-rw-rw---- 1 mysql mysql       8948 Sep 10 18:20 nagios_processevents.frm
-rw-rw---- 1 mysql mysql      85324 Jan 15 06:31 nagios_processevents.MYD
-rw-rw---- 1 mysql mysql      19456 Jan 15 06:40 nagios_processevents.MYI
-rw-rw---- 1 mysql mysql      10174 Sep 10 18:20 nagios_programstatus.frm
-rw-rw---- 1 mysql mysql        132 Jan 15 07:29 nagios_programstatus.MYD
-rw-rw---- 1 mysql mysql       3072 Jan 15 07:35 nagios_programstatus.MYI
-rw-rw---- 1 mysql mysql       8706 Sep 10 18:20 nagios_runtimevariables.frm
-rw-rw---- 1 mysql mysql        808 Jan 15 06:40 nagios_runtimevariables.MYD
-rw-rw---- 1 mysql mysql       3072 Jan 15 06:40 nagios_runtimevariables.MYI
-rw-rw---- 1 mysql mysql       9332 Sep 10 18:20 nagios_scheduleddowntime.frm
-rw-rw---- 1 mysql mysql       7432 Jan 15 06:40 nagios_scheduleddowntime.MYD
-rw-rw---- 1 mysql mysql       5120 Jan 15 06:40 nagios_scheduleddowntime.MYI
-rw-rw---- 1 mysql mysql       9626 Sep 10 18:20 nagios_servicechecks.frm
-rw-rw---- 1 mysql mysql          0 Sep 10 18:20 nagios_servicechecks.MYD
-rw-rw---- 1 mysql mysql       1024 Jan 15 06:40 nagios_servicechecks.MYI
-rw-rw---- 1 mysql mysql       8750 Sep 10 18:20 nagios_service_contactgroups.frm
-rw-rw---- 1 mysql mysql      19590 Jan 15 06:30 nagios_service_contactgroups.MYD
-rw-rw---- 1 mysql mysql      34816 Jan 15 06:40 nagios_service_contactgroups.MYI
-rw-rw---- 1 mysql mysql       8730 Sep 10 18:20 nagios_service_contacts.frm
-rw-rw---- 1 mysql mysql       3480 Jan 15 06:30 nagios_service_contacts.MYD
-rw-rw---- 1 mysql mysql      11264 Jan 15 06:40 nagios_service_contacts.MYI
-rw-rw---- 1 mysql mysql       9178 Sep 10 18:20 nagios_servicedependencies.frm
-rw-rw---- 1 mysql mysql          0 Sep 10 18:20 nagios_servicedependencies.MYD
-rw-rw---- 1 mysql mysql       1024 Jan 15 06:40 nagios_servicedependencies.MYI
-rw-rw---- 1 mysql mysql       8790 Sep 10 18:20 nagios_serviceescalation_contactgroups.frm
-rw-rw---- 1 mysql mysql          0 Sep 10 18:20 nagios_serviceescalation_contactgroups.MYD
-rw-rw---- 1 mysql mysql       1024 Jan 15 06:40 nagios_serviceescalation_contactgroups.MYI
-rw-rw---- 1 mysql mysql       8770 Sep 10 18:20 nagios_serviceescalation_contacts.frm
-rw-rw---- 1 mysql mysql          0 Sep 10 18:20 nagios_serviceescalation_contacts.MYD
-rw-rw---- 1 mysql mysql       1024 Jan 15 06:40 nagios_serviceescalation_contacts.MYI
-rw-rw---- 1 mysql mysql       9220 Sep 10 18:20 nagios_serviceescalations.frm
-rw-rw---- 1 mysql mysql          0 Sep 10 18:20 nagios_serviceescalations.MYD
-rw-rw---- 1 mysql mysql       1024 Jan 15 06:40 nagios_serviceescalations.MYI
-rw-rw---- 1 mysql mysql       8748 Sep 10 18:20 nagios_servicegroup_members.frm
-rw-rw---- 1 mysql mysql         75 Jan 15 06:30 nagios_servicegroup_members.MYD
-rw-rw---- 1 mysql mysql       3072 Jan 15 06:40 nagios_servicegroup_members.MYI
-rw-rw---- 1 mysql mysql       8768 Sep 10 18:20 nagios_servicegroups.frm
-rw-rw---- 1 mysql mysql        156 Jan 15 06:30 nagios_servicegroups.MYD
-rw-rw---- 1 mysql mysql       3072 Jan 15 06:40 nagios_servicegroups.MYI
-rw-rw---- 1 mysql mysql       8756 Sep 10 18:20 nagios_service_parentservices.frm
-rw-rw---- 1 mysql mysql          0 Sep 10 18:20 nagios_service_parentservices.MYD
-rw-rw---- 1 mysql mysql       1024 Sep 27 22:19 nagios_service_parentservices.MYI
-rw-rw---- 1 mysql mysql      11766 Sep 10 18:20 nagios_services.frm
-rw-rw---- 1 mysql mysql     971272 Jan 15 06:40 nagios_services.MYD
-rw-rw---- 1 mysql mysql     207872 Jan 15 06:40 nagios_services.MYI
-rw-rw---- 1 mysql mysql      19544 Sep 10 18:20 nagios_servicestatus.frm
-rw-rw---- 1 mysql mysql    2599604 Jan 15 07:29 nagios_servicestatus.MYD
-rw-rw---- 1 mysql mysql    1215488 Jan 15 07:35 nagios_servicestatus.MYI
-rw-rw---- 1 mysql mysql       9174 Sep 10 18:20 nagios_statehistory.frm
-rw-rw---- 1 mysql mysql 1242660864 Jan 15 07:35 nagios_statehistory.MYD
-rw-rw---- 1 mysql mysql  283782144 Jan 15 07:35 nagios_statehistory.MYI
-rw-rw---- 1 mysql mysql       9110 Sep 10 18:20 nagios_systemcommands.frm
-rw-rw---- 1 mysql mysql      55736 Jan 15 07:29 nagios_systemcommands.MYD
-rw-rw---- 1 mysql mysql      14336 Jan 15 07:35 nagios_systemcommands.MYI
-rw-rw---- 1 mysql mysql       8914 Sep 10 18:20 nagios_timedeventqueue.frm
-rw-rw---- 1 mysql mysql          0 Sep 10 18:20 nagios_timedeventqueue.MYD
-rw-rw---- 1 mysql mysql       1024 Jan 15 06:40 nagios_timedeventqueue.MYI
-rw-rw---- 1 mysql mysql       9104 Sep 10 18:20 nagios_timedevents.frm
-rw-rw---- 1 mysql mysql          0 Sep 10 18:20 nagios_timedevents.MYD
-rw-rw---- 1 mysql mysql       1024 Jan 15 06:40 nagios_timedevents.MYI
-rw-rw---- 1 mysql mysql       8760 Sep 10 18:20 nagios_timeperiods.frm
-rw-rw---- 1 mysql mysql      21292 Jan 15 06:40 nagios_timeperiods.MYD
-rw-rw---- 1 mysql mysql      14336 Jan 15 06:40 nagios_timeperiods.MYI
-rw-rw---- 1 mysql mysql       8794 Sep 10 18:20 nagios_timeperiod_timeranges.frm
-rw-rw---- 1 mysql mysql      62244 Jan 15 06:30 nagios_timeperiod_timeranges.MYD
-rw-rw---- 1 mysql mysql      96256 Jan 15 06:40 nagios_timeperiod_timeranges.MYI
/var/lib/pgsql/data/base/16385

Code: Select all

-rw------- 1 postgres postgres     278528 Jan 15 08:33 12661
-rw------- 1 postgres postgres      24576 Jan 15 07:29 12661_fsm
-rw------- 1 postgres postgres       8192 Jan 14 09:53 12661_vm
-rw------- 1 postgres postgres     352256 Jan 15 07:14 12663
-rw------- 1 postgres postgres      24576 Jan 15 07:04 12663_fsm
-rw------- 1 postgres postgres       8192 Jan 15 07:04 12663_vm
-rw------- 1 postgres postgres      16384 Jan 15 07:14 12665
-rw------- 1 postgres postgres      40960 Jan 15 07:29 12666
-rw------- 1 postgres postgres      98304 Sep 10 18:25 12667
-rw------- 1 postgres postgres      24576 Sep 10 18:26 12667_fsm
-rw------- 1 postgres postgres       8192 Sep 10 18:26 12667_vm
-rw------- 1 postgres postgres      32768 Sep 10 18:25 12669
-rw------- 1 postgres postgres      32768 Sep 10 18:26 12670
-rw------- 1 postgres postgres          0 Jul  3  2018 12675
-rw------- 1 postgres postgres       8192 Jul  3  2018 12677
-rw------- 1 postgres postgres       8192 Jul  3  2018 12678
-rw------- 1 postgres postgres     442368 Sep 10 18:26 12679
-rw------- 1 postgres postgres      24576 Sep 10 18:26 12679_fsm
-rw------- 1 postgres postgres       8192 Sep 10 18:26 12679_vm
-rw------- 1 postgres postgres     180224 Sep 10 18:26 12681
-rw------- 1 postgres postgres      24576 Sep 10 18:26 12681_fsm
-rw------- 1 postgres postgres     131072 Sep 10 18:26 12682
-rw------- 1 postgres postgres      24576 Sep 10 18:26 12682_fsm
-rw------- 1 postgres postgres     507904 Jul  3  2018 12683
-rw------- 1 postgres postgres      24576 Jul  3  2018 12683_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12683_vm
-rw------- 1 postgres postgres          0 Jul  3  2018 12685
-rw------- 1 postgres postgres       8192 Jul  3  2018 12687
-rw------- 1 postgres postgres      73728 Jul  3  2018 12688
-rw------- 1 postgres postgres     196608 Jul  3  2018 12689
-rw------- 1 postgres postgres      98304 Jan 15 08:29 12690
-rw------- 1 postgres postgres      24576 Sep 10 18:26 12690_fsm
-rw------- 1 postgres postgres       8192 Sep 10 18:26 12690_vm
-rw------- 1 postgres postgres      32768 Sep 10 18:25 12692
-rw------- 1 postgres postgres      49152 Sep 10 18:25 12693
-rw------- 1 postgres postgres      40960 Sep 10 18:25 12694
-rw------- 1 postgres postgres      24576 Sep 10 18:26 12694_fsm
-rw------- 1 postgres postgres       8192 Sep 10 18:26 12694_vm
-rw------- 1 postgres postgres          0 Jul  3  2018 12696
-rw------- 1 postgres postgres       8192 Jul  3  2018 12698
-rw------- 1 postgres postgres      16384 Sep 10 18:25 12699
-rw------- 1 postgres postgres      16384 Sep 10 18:21 12700
-rw------- 1 postgres postgres      16384 Sep 11 07:01 12701
-rw------- 1 postgres postgres      24576 Sep 10 18:20 12701_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12701_vm
-rw------- 1 postgres postgres          0 Jul  3  2018 12703
-rw------- 1 postgres postgres       8192 Jul  3  2018 12705
-rw------- 1 postgres postgres      16384 Sep 10 18:20 12706
-rw------- 1 postgres postgres      16384 Sep 10 18:20 12707
-rw------- 1 postgres postgres      16384 Sep 10 18:20 12708
-rw------- 1 postgres postgres      16384 Sep 10 18:20 12709
-rw------- 1 postgres postgres          0 Jul  3  2018 12710
-rw------- 1 postgres postgres       8192 Jul  3  2018 12712
-rw------- 1 postgres postgres       8192 Jul  3  2018 12713
-rw------- 1 postgres postgres      32768 Sep 10 18:25 12714
-rw------- 1 postgres postgres      24576 Sep 10 18:26 12714_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12714_vm
-rw------- 1 postgres postgres      16384 Sep 10 18:25 12716
-rw------- 1 postgres postgres      16384 Sep 10 18:25 12717
-rw------- 1 postgres postgres     114688 Jul  3  2018 12718
-rw------- 1 postgres postgres      24576 Jul  3  2018 12718_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12718_vm
-rw------- 1 postgres postgres      32768 Jul  3  2018 12720
-rw------- 1 postgres postgres      40960 Jul  3  2018 12721
-rw------- 1 postgres postgres      16384 Jul  3  2018 12722
-rw------- 1 postgres postgres      24576 Jul  3  2018 12722_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12722_vm
-rw------- 1 postgres postgres      16384 Jul  3  2018 12724
-rw------- 1 postgres postgres      16384 Jul  3  2018 12725
-rw------- 1 postgres postgres      16384 Jul  3  2018 12726
-rw------- 1 postgres postgres      24576 Jul  3  2018 12726_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12726_vm
-rw------- 1 postgres postgres      16384 Jul  3  2018 12728
-rw------- 1 postgres postgres      16384 Jul  3  2018 12729
-rw------- 1 postgres postgres       8192 Jul  3  2018 12730
-rw------- 1 postgres postgres      24576 Jul  3  2018 12730_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12730_vm
-rw------- 1 postgres postgres      16384 Jul  3  2018 12732
-rw------- 1 postgres postgres      16384 Jul  3  2018 12733
-rw------- 1 postgres postgres      32768 Jul  3  2018 12734
-rw------- 1 postgres postgres      24576 Jul  3  2018 12734_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12734_vm
-rw------- 1 postgres postgres      32768 Jul  3  2018 12736
-rw------- 1 postgres postgres      32768 Jul  3  2018 12737
-rw------- 1 postgres postgres      32768 Jul  3  2018 12738
-rw------- 1 postgres postgres      24576 Jul  3  2018 12739
-rw------- 1 postgres postgres      24576 Jul  3  2018 12739_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12739_vm
-rw------- 1 postgres postgres      32768 Jul  3  2018 12741
-rw------- 1 postgres postgres      16384 Jul  3  2018 12742
-rw------- 1 postgres postgres       8192 Jul  3  2018 12743
-rw------- 1 postgres postgres      24576 Jul  3  2018 12743_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12743_vm
-rw------- 1 postgres postgres      16384 Jul  3  2018 12745
-rw------- 1 postgres postgres      16384 Jul  3  2018 12746
-rw------- 1 postgres postgres          0 Jul  3  2018 12747
-rw------- 1 postgres postgres       8192 Jul  3  2018 12749
-rw------- 1 postgres postgres       8192 Jul  3  2018 12750
-rw------- 1 postgres postgres      24576 Jul  3  2018 12750_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12750_vm
-rw------- 1 postgres postgres      16384 Jul  3  2018 12752
-rw------- 1 postgres postgres      98304 Jul  3  2018 12753
-rw------- 1 postgres postgres      24576 Jul  3  2018 12753_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12753_vm
-rw------- 1 postgres postgres     327680 Jul  3  2018 12755
-rw------- 1 postgres postgres      24576 Jul  3  2018 12755_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12755_vm
-rw------- 1 postgres postgres      16384 Jul  3  2018 12757
-rw------- 1 postgres postgres      16384 Jul  3  2018 12758
-rw------- 1 postgres postgres      16384 Jul  3  2018 12759
-rw------- 1 postgres postgres          0 Jul  3  2018 12760
-rw------- 1 postgres postgres          0 Jul  3  2018 12762
-rw------- 1 postgres postgres       8192 Jul  3  2018 12764
-rw------- 1 postgres postgres       8192 Jul  3  2018 12765
-rw------- 1 postgres postgres       8192 Jul  3  2018 12766
-rw------- 1 postgres postgres       8192 Jul  3  2018 12767
-rw------- 1 postgres postgres     245760 Jul  3  2018 12768
-rw------- 1 postgres postgres      24576 Jul  3  2018 12768_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12768_vm
-rw------- 1 postgres postgres          0 Jul  3  2018 12770
-rw------- 1 postgres postgres       8192 Jul  3  2018 12772
-rw------- 1 postgres postgres     131072 Jul  3  2018 12773
-rw------- 1 postgres postgres      16384 Jul  3  2018 12774
-rw------- 1 postgres postgres      24576 Jul  3  2018 12774_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12774_vm
-rw------- 1 postgres postgres      16384 Jul  3  2018 12776
-rw------- 1 postgres postgres      16384 Jul  3  2018 12777
-rw------- 1 postgres postgres          0 Jul  3  2018 12778
-rw------- 1 postgres postgres       8192 Jul  3  2018 12780
-rw------- 1 postgres postgres       8192 Jul  3  2018 12781
-rw------- 1 postgres postgres       8192 Jul  3  2018 12782
-rw------- 1 postgres postgres       8192 Sep 10 18:20 12783
-rw------- 1 postgres postgres      24576 Jul  3  2018 12783_fsm
-rw------- 1 postgres postgres       8192 Sep  8 16:00 12783_vm
-rw------- 1 postgres postgres      16384 Jul  3  2018 12785
-rw------- 1 postgres postgres      16384 Jul  3  2018 12786
-rw------- 1 postgres postgres      24576 Jul  3  2018 12787
-rw------- 1 postgres postgres      24576 Jul  3  2018 12787_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12787_vm
-rw------- 1 postgres postgres      16384 Jul  3  2018 12789
-rw------- 1 postgres postgres      16384 Jul  3  2018 12790
-rw------- 1 postgres postgres      16384 Jul  3  2018 12791
-rw------- 1 postgres postgres     409600 Sep 11 07:01 12792
-rw------- 1 postgres postgres      24576 Jul  3  2018 12792_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12792_vm
-rw------- 1 postgres postgres     253952 Sep 10 18:20 12794
-rw------- 1 postgres postgres     253952 Sep 10 18:20 12795
-rw------- 1 postgres postgres       8192 Jul  3  2018 12827
-rw------- 1 postgres postgres      24576 Jul  3  2018 12827_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12827_vm
-rw------- 1 postgres postgres      16384 Jul  3  2018 12829
-rw------- 1 postgres postgres      16384 Jul  3  2018 12830
-rw------- 1 postgres postgres      16384 Jul  3  2018 12831
-rw------- 1 postgres postgres      24576 Jul  3  2018 12831_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12831_vm
-rw------- 1 postgres postgres      32768 Jul  3  2018 12833
-rw------- 1 postgres postgres       8192 Jul  3  2018 12834
-rw------- 1 postgres postgres      24576 Jul  3  2018 12834_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12834_vm
-rw------- 1 postgres postgres      16384 Jul  3  2018 12836
-rw------- 1 postgres postgres      16384 Jul  3  2018 12837
-rw------- 1 postgres postgres       8192 Jul  3  2018 12838
-rw------- 1 postgres postgres      24576 Jul  3  2018 12838_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12838_vm
-rw------- 1 postgres postgres      16384 Jul  3  2018 12840
-rw------- 1 postgres postgres      16384 Jul  3  2018 12841
-rw------- 1 postgres postgres       8192 Jul  3  2018 12842
-rw------- 1 postgres postgres      24576 Jul  3  2018 12842_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12842_vm
-rw------- 1 postgres postgres      16384 Jul  3  2018 12844
-rw------- 1 postgres postgres      16384 Jul  3  2018 12845
-rw------- 1 postgres postgres       8192 Jul  3  2018 12846
-rw------- 1 postgres postgres      24576 Jul  3  2018 12846_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12846_vm
-rw------- 1 postgres postgres      16384 Jul  3  2018 12848
-rw------- 1 postgres postgres      16384 Jul  3  2018 12849
-rw------- 1 postgres postgres          0 Jul  3  2018 12850
-rw------- 1 postgres postgres       8192 Jul  3  2018 12852
-rw------- 1 postgres postgres       8192 Jul  3  2018 12853
-rw------- 1 postgres postgres          0 Jul  3  2018 12854
-rw------- 1 postgres postgres       8192 Jul  3  2018 12856
-rw------- 1 postgres postgres       8192 Jul  3  2018 12857
-rw------- 1 postgres postgres          0 Jul  3  2018 12858
-rw------- 1 postgres postgres       8192 Jul  3  2018 12860
-rw------- 1 postgres postgres          0 Jul  3  2018 12861
-rw------- 1 postgres postgres       8192 Jul  3  2018 12863
-rw------- 1 postgres postgres       8192 Jul  3  2018 12864
-rw------- 1 postgres postgres          0 Jul  3  2018 12865
-rw------- 1 postgres postgres          0 Jul  3  2018 12867
-rw------- 1 postgres postgres       8192 Jul  3  2018 12869
-rw------- 1 postgres postgres       8192 Jul  3  2018 12870
-rw------- 1 postgres postgres     229376 Jul  3  2018 12874
-rw------- 1 postgres postgres      24576 Jul  3  2018 12874_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12874_vm
-rw------- 1 postgres postgres      40960 Jul  3  2018 12876
-rw------- 1 postgres postgres      57344 Jul  3  2018 12877
-rw------- 1 postgres postgres       8192 Jul  3  2018 12878
-rw------- 1 postgres postgres      24576 Jul  3  2018 12878_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12878_vm
-rw------- 1 postgres postgres      16384 Jul  3  2018 12880
-rw------- 1 postgres postgres          0 Jul  3  2018 12881
-rw------- 1 postgres postgres       8192 Jul  3  2018 12883
-rw------- 1 postgres postgres      57344 Jul  3  2018 12884
-rw------- 1 postgres postgres      24576 Jul  3  2018 12884_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12884_vm
-rw------- 1 postgres postgres          0 Jul  3  2018 12886
-rw------- 1 postgres postgres       8192 Jul  3  2018 12888
-rw------- 1 postgres postgres       8192 Jul  3  2018 12889
-rw------- 1 postgres postgres      24576 Jul  3  2018 12889_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12889_vm
-rw------- 1 postgres postgres          0 Jul  3  2018 12891
-rw------- 1 postgres postgres       8192 Jul  3  2018 12893
-rw------- 1 postgres postgres       8192 Jul  3  2018 12894
-rw------- 1 postgres postgres      24576 Jul  3  2018 12894_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12894_vm
-rw------- 1 postgres postgres          0 Jul  3  2018 12896
-rw------- 1 postgres postgres       8192 Jul  3  2018 12898
-rw------- 1 postgres postgres       8192 Jul  3  2018 12899
-rw------- 1 postgres postgres      24576 Jul  3  2018 12899_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12899_vm
-rw------- 1 postgres postgres          0 Jul  3  2018 12901
-rw------- 1 postgres postgres       8192 Jul  3  2018 12903
-rw------- 1 postgres postgres       8192 Jul  3  2018 12904
-rw------- 1 postgres postgres      24576 Jul  3  2018 12904_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12904_vm
-rw------- 1 postgres postgres          0 Jul  3  2018 12906
-rw------- 1 postgres postgres       8192 Jul  3  2018 12908
-rw------- 1 postgres postgres       8192 Jul  3  2018 12909
-rw------- 1 postgres postgres      24576 Jul  3  2018 12909_fsm
-rw------- 1 postgres postgres       8192 Jul  3  2018 12909_vm
-rw------- 1 postgres postgres          0 Jul  3  2018 12911
-rw------- 1 postgres postgres       8192 Jul  3  2018 12913
-rw------- 1 postgres postgres          0 Jul  3  2018 12914
-rw------- 1 postgres postgres          0 Jul  3  2018 12916
-rw------- 1 postgres postgres       8192 Jul  3  2018 12918
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66220
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66222
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66224
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66226
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66228
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66230
-rw------- 1 postgres postgres     663552 Jan 15 07:14 66232
-rw------- 1 postgres postgres      24576 Jan 14 09:54 66232_fsm
-rw------- 1 postgres postgres       8192 Jan 15 07:14 66232_vm
-rw------- 1 postgres postgres      16384 Dec 17 10:42 66235
-rw------- 1 postgres postgres      24576 Dec 17 10:42 66235_fsm
-rw------- 1 postgres postgres       8192 Dec 17 10:42 66235_vm
-rw------- 1 postgres postgres      16384 Dec 17 10:42 66237
-rw------- 1 postgres postgres       8192 Jan 15 06:27 66238
-rw------- 1 postgres postgres       8192 Jan 15 08:29 66240
-rw------- 1 postgres postgres     671744 Jan 15 08:29 66242
-rw------- 1 postgres postgres      24576 Jan 15 08:29 66242_fsm
-rw------- 1 postgres postgres       8192 Jan 15 08:29 66242_vm
-rw------- 1 postgres postgres          0 Sep 10 18:20 66247
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66249
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66250
-rw------- 1 postgres postgres          0 Sep 10 18:20 66252
-rw------- 1 postgres postgres          0 Sep 10 18:20 66258
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66260
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66261
-rw------- 1 postgres postgres          0 Sep 10 18:20 66263
-rw------- 1 postgres postgres          0 Sep 10 18:20 66267
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66269
-rw------- 1 postgres postgres       8192 Jan 15 06:27 66270
-rw------- 1 postgres postgres      16384 Jan 15 08:29 66272
-rw------- 1 postgres postgres      24576 Jan 15 07:14 66272_fsm
-rw------- 1 postgres postgres       8192 Jan 15 08:31 66272_vm
-rw------- 1 postgres postgres          0 Sep 10 18:20 66284
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66286
-rw------- 1 postgres postgres       8192 Jan 15 08:30 66287
-rw------- 1 postgres postgres  185294848 Jan 15 08:29 66289
-rw------- 1 postgres postgres      65536 Jan 15 08:29 66289_fsm
-rw------- 1 postgres postgres          0 Jan 11 19:38 66289_vm
-rw------- 1 postgres postgres     237568 Jan 15 05:28 66294
-rw------- 1 postgres postgres      24576 Jan 15 05:28 66294_fsm
-rw------- 1 postgres postgres          0 Jan 11 13:35 66294_vm
-rw------- 1 postgres postgres      16384 Jan 15 05:28 66296
-rw------- 1 postgres postgres       8192 Jan 15 08:29 66297
-rw------- 1 postgres postgres  127967232 Jan 15 08:31 66299
-rw------- 1 postgres postgres      49152 Jan 15 08:29 66299_fsm
-rw------- 1 postgres postgres       8192 Jan 15 08:29 66299_vm
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66305
-rw------- 1 postgres postgres          0 Sep 10 18:20 66307
-rw------- 1 postgres postgres          0 Sep 10 18:20 66313
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66315
-rw------- 1 postgres postgres       8192 Jan 15 08:29 66316
-rw------- 1 postgres postgres 1073741824 Jan 15 08:31 66318
-rw------- 1 postgres postgres 1073741824 Jan 15 07:26 66318.1
-rw------- 1 postgres postgres 1073741824 Jan 15 05:53 66318.2
-rw------- 1 postgres postgres 1073741824 Jan 15 06:45 66318.3
-rw------- 1 postgres postgres 1073741824 Jan 15 07:08 66318.4
-rw------- 1 postgres postgres  876052480 Jan 15 08:31 66318.5
-rw------- 1 postgres postgres    1556480 Jan 15 08:31 66318_fsm
-rw------- 1 postgres postgres      32768 Jan 15 08:30 66318_vm
-rw------- 1 postgres postgres    8060928 Jan 15 08:30 66324
-rw------- 1 postgres postgres      24576 Jan 15 08:29 66324_fsm
-rw------- 1 postgres postgres       8192 Jan 15 06:58 66324_vm
-rw------- 1 postgres postgres     122880 Jan 15 08:29 66326
-rw------- 1 postgres postgres       8192 Jan  7 06:53 66327
-rw------- 1 postgres postgres      16384 Jan 15 07:14 66329
-rw------- 1 postgres postgres      24576 Jan  7 06:58 66329_fsm
-rw------- 1 postgres postgres       8192 Jan 15 07:14 66329_vm
-rw------- 1 postgres postgres          0 Sep 10 18:20 66333
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66335
-rw------- 1 postgres postgres       8192 Jan 15 05:21 66336
-rw------- 1 postgres postgres       8192 Jan 15 08:07 66338
-rw------- 1 postgres postgres      24576 Jan 15 08:29 66338_fsm
-rw------- 1 postgres postgres       8192 Jan 14 05:37 66338_vm
-rw------- 1 postgres postgres          0 Sep 10 18:20 66342
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66344
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66345
-rw------- 1 postgres postgres     106496 Jan 15 08:30 66347
-rw------- 1 postgres postgres      24576 Jan 15 08:31 66347_fsm
-rw------- 1 postgres postgres       8192 Jan 15 08:31 66347_vm
-rw------- 1 postgres postgres          0 Sep 10 18:20 66351
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66353
-rw------- 1 postgres postgres       8192 Jan  7 08:58 66354
-rw------- 1 postgres postgres     524288 Jan 15 08:30 66356
-rw------- 1 postgres postgres      24576 Jan  7 09:03 66356_fsm
-rw------- 1 postgres postgres       8192 Jan 15 08:28 66356_vm
-rw------- 1 postgres postgres          0 Sep 10 18:20 66361
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66363
-rw------- 1 postgres postgres       8192 Jan  2 09:35 66364
-rw------- 1 postgres postgres     114688 Jan 15 08:29 66366
-rw------- 1 postgres postgres      24576 Jan  7 10:18 66366_fsm
-rw------- 1 postgres postgres       8192 Jan 15 08:28 66366_vm
-rw------- 1 postgres postgres     425984 Jan 15 06:27 66381
-rw------- 1 postgres postgres      24576 Jan 14 05:57 66381_fsm
-rw------- 1 postgres postgres     761856 Jan 15 08:29 66383
-rw------- 1 postgres postgres      24576 Jan 15 08:29 66383_fsm
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66385
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66387
-rw------- 1 postgres postgres       8192 Sep 10 18:20 66389
-rw------- 1 postgres postgres      57344 Jan 15 07:14 66391
-rw------- 1 postgres postgres      24576 Dec 17 16:57 66391_fsm
-rw------- 1 postgres postgres   48570368 Jan 15 08:31 66393
-rw------- 1 postgres postgres      32768 Jan 15 08:29 66393_fsm
-rw------- 1 postgres postgres      16384 Jan  7 06:53 66395
-rw------- 1 postgres postgres      16384 Jan 15 08:29 66397
-rw------- 1 postgres postgres      16384 Jan 15 07:28 66399
-rw------- 1 postgres postgres     147456 Jan  7 08:58 66401
-rw------- 1 postgres postgres      24576 Dec 17 08:06 66401_fsm
-rw------- 1 postgres postgres     278528 Jan  7 08:58 66403
-rw------- 1 postgres postgres      24576 Dec 17 08:06 66403_fsm
-rw------- 1 postgres postgres      16384 Jan  2 09:36 66405
-rw------- 1 postgres postgres      32768 Jan  2 09:35 66407
-rw------- 1 postgres postgres   61366272 Jan 15 08:31 66409
-rw------- 1 postgres postgres      32768 Jan 15 08:29 66409_fsm
-rw------- 1 postgres postgres     737280 Jan 15 06:27 66410
-rw------- 1 postgres postgres      24576 Jan  9 02:24 66410_fsm
-rw------- 1 postgres postgres     425984 Jan 15 06:27 66411
-rw------- 1 postgres postgres      24576 Jan  8 05:34 66411_fsm
-rw------- 1 postgres postgres     737280 Jan 15 06:27 66412
-rw------- 1 postgres postgres      24576 Jan 10 06:39 66412_fsm
-rw------- 1 postgres postgres     565248 Jan 15 06:27 66413
-rw------- 1 postgres postgres      24576 Jan 12 01:12 66413_fsm
-rw------- 1 postgres postgres     720896 Jan 15 06:27 66414
-rw------- 1 postgres postgres      24576 Jan  9 02:19 66414_fsm
-rw------- 1 postgres postgres      57344 Jan 15 07:14 66415
-rw------- 1 postgres postgres      24576 Dec 17 16:57 66415_fsm
-rw------- 1 postgres postgres  211664896 Jan 15 08:31 66416
-rw------- 1 postgres postgres      24576 Jan 15 05:08 66416_fsm
-rw------- 1 postgres postgres  123224064 Jan 15 08:31 66417
-rw------- 1 postgres postgres      24576 Jan 15 05:08 66417_fsm
-rw------- 1 postgres postgres  155885568 Jan 15 08:31 66418
-rw------- 1 postgres postgres      24576 Jan 15 05:08 66418_fsm
-rw------- 1 postgres postgres      16384 Jan  7 06:53 66419
-rw------- 1 postgres postgres      16384 Jan 15 07:28 66420
-rw------- 1 postgres postgres     147456 Jan  7 08:58 66421
-rw------- 1 postgres postgres      24576 Dec 17 08:06 66421_fsm
-rw------- 1 postgres postgres        512 Jul  3  2018 pg_filenode.map
-rw------- 1 postgres postgres     111220 Jan 15 07:26 pg_internal.init
-rw------- 1 postgres postgres      98304 Jan 15 07:14 pg_internal.init.72949
-rw------- 1 postgres postgres          0 Jan 15 07:14 pg_internal.init.73145
-rw------- 1 postgres postgres          0 Jan 15 07:14 pg_internal.init.73155
-rw------- 1 postgres postgres          0 Jan 15 07:14 pg_internal.init.73160
-rw------- 1 postgres postgres          0 Jan 15 07:14 pg_internal.init.73192
-rw------- 1 postgres postgres          0 Jan 15 07:14 pg_internal.init.73218
-rw------- 1 postgres postgres          0 Jan 15 07:14 pg_internal.init.73219
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73238
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73244
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73295
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73297
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73303
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73304
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73305
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73306
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73307
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73328
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73329
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73345
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73346
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73351
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73352
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73355
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73377
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73378
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73380
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73389
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73391
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73392
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73393
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73394
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73402
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73403
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73405
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73406
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73407
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73408
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73409
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73410
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73411
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73412
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73413
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73414
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73415
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73416
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73417
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73418
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73419
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73420
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73421
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73422
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73423
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73424
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73425
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73426
-rw------- 1 postgres postgres          0 Jan 15 07:15 pg_internal.init.73430

Code: Select all

bash-4.2$ df -i
Filesystem                        Inodes  IUsed    IFree IUse% Mounted on
/dev/mapper/VolGroup00-LogVol00 29358080 171773 29186307    1% /
devtmpfs                          994726    380   994346    1% /dev
tmpfs                             998780      1   998779    1% /dev/shm
tmpfs                             998780   1467   997313    1% /run
tmpfs                             998780     16   998764    1% /sys/fs/cgroup
/dev/sda1                         262144    342   261802    1% /boot
/dev/mapper/VolGroup00-LogVol01    12648   8905     3743   71% /var
/dev/mapper/VolGroupU1-u01      83884032   9482 83874550    1% /u01
/dev/mapper/VolGroup00-LogVol03  4194304    124  4194180    1% /tmp
tmpfs                             998780     10   998770    1% /var/nagiosramdisk
13.129.19.59:/img_kdump         62975998  39208 62936790    1% /kdump
tmpfs                             998780      1   998779    1% /run/user/0
tmpfs                             998780      1   998779    1% /run/user/103008
usa0300ux448:/HOME_DIR/50004256  6297598 470562  5827036    8% /home/50004256
usa0300ux448:/HOME_DIR/q5c9nf32  6297598 470562  5827036    8% /home/q5c9nf32
tmpfs                             998780      1   998779    1% /run/user/103032
usa0300ux448:/HOME_DIR/q5c9nf3f  6297598 470562  5827036    8% /home/q5c9nf3f
usa0300ux448:/HOME_DIR/nagios    6297598 470562  5827036    8% /home/nagios

Code: Select all

Filesystem                       Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00   56G   39G   18G  69% /
devtmpfs                         3.8G     0  3.8G   0% /dev
tmpfs                            3.9G     0  3.9G   0% /dev/shm
tmpfs                            3.9G  404M  3.5G  11% /run
tmpfs                            3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/sda1                        509M  270M  240M  53% /boot
/dev/mapper/VolGroup00-LogVol01   13G   13G   20K 100% /var
/dev/mapper/VolGroupU1-u01       160G  388M  160G   1% /u01
/dev/mapper/VolGroup00-LogVol03  8.0G  375M  7.7G   5% /tmp
tmpfs                            500M   18M  483M   4% /var/nagiosramdisk
13.129.19.59:/img_kdump          493G   44G  450G   9% /kdump
tmpfs                            781M     0  781M   0% /run/user/0
tmpfs                            781M     0  781M   0% /run/user/103008
usa0300ux448:/HOME_DIR/50004256   50G   42G  7.5G  85% /home/50004256
usa0300ux448:/HOME_DIR/q5c9nf32   50G   42G  7.5G  85% /home/q5c9nf32
tmpfs                            781M     0  781M   0% /run/user/103032
usa0300ux448:/HOME_DIR/q5c9nf3f   50G   42G  7.5G  85% /home/q5c9nf3f
usa0300ux448:/HOME_DIR/nagios     50G   42G  7.5G  85% /home/nagios

Code: Select all

bash-4.2$ sudo ./repair_databases.sh
DATABASE: nagios
TABLE:
/var/lib/mysql/nagios /usr/local/nagiosxi/scripts
- recovering (with sort) MyISAM-table 'nagios_acknowledgements'
Data records: 49
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_commands'
Data records: 441
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_commenthistory'
Data records: 108166
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_comments'
Data records: 131
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_configfiles'
Data records: 1
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_configfilevariables'
Data records: 132
- Fixing index 1

---------

- recovering (with sort) MyISAM-table 'nagios_conninfo'
Data records: 571
- Fixing index 1

---------

- recovering (with sort) MyISAM-table 'nagios_contact_addresses'
Data records: 0
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_contactgroup_members'
Data records: 909
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_contactgroups'
Data records: 284
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_contact_notificationcommands'
Data records: 1464
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_contactnotificationmethods'
Data records: 97039
- Fixing index 1
- Fixing index 2
- Fixing index 3

---------

- recovering (with sort) MyISAM-table 'nagios_contactnotifications'
Data records: 97038
- Fixing index 1
- Fixing index 2
- Fixing index 3
- Fixing index 4

---------

- recovering (with sort) MyISAM-table 'nagios_contacts'
Data records: 122
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_contactstatus'
Data records: 122
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_customvariables'
Data records: 8247
- Fixing index 1
- Fixing index 2
- Fixing index 3

---------

- recovering (with sort) MyISAM-table 'nagios_customvariablestatus'
Data records: 8247
- Fixing index 1
- Fixing index 2
- Fixing index 3

---------

- recovering (with keycache) MyISAM-table 'nagios_dbversion'
Data records: 1

---------

- recovering (with sort) MyISAM-table 'nagios_downtimehistory'
Data records: 2196
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_eventhandlers'
Data records: 233
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_externalcommands'
Data records: 76
- Fixing index 1

---------

- recovering (with sort) MyISAM-table 'nagios_flappinghistory'
Data records: 170890
- Fixing index 1

---------

- recovering (with sort) MyISAM-table 'nagios_hostchecks'
Data records: 0
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_host_contactgroups'
Data records: 113
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_host_contacts'
Data records: 7
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_hostdependencies'
Data records: 0
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_hostescalation_contactgroups'
Data records: 0
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_hostescalation_contacts'
Data records: 0
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_hostescalations'
Data records: 0
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_hostgroup_members'
Data records: 2745
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_hostgroups'
Data records: 625
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_host_parenthosts'
Data records: 0
- Fixing index 1
- Fixing index 2

---------

- recovering (with sort) MyISAM-table 'nagios_hosts'
Data records: 804
- Fixing index 1
- Fixing index 2
- Fixing index 3

---------

- recovering (with sort) MyISAM-table 'nagios_hoststatus'
Data records: 804
- Fixing index 1
- Fixing index 2
- Fixing index 3
- Fixing index 4
- Fixing index 5
- Fixing index 6
- Fixing index 7
- Fixing index 8
- Fixing index 9
- Fixing index 10
- Fixing index 11
- Fixing index 12
- Fixing index 13
- Fixing index 14
- Fixing index 15
- Fixing index 16
- Fixing index 17
- Fixing index 18
- Fixing index 19

---------

- recovering (with sort) MyISAM-table 'nagios_instances'
Data records: 1
- Fixing index 1

---------

- recovering (with sort) MyISAM-table 'nagios_logentries'
Data records: 7337497
- Fixing index 1
- Fixing index 2
- Fixing index 3
- Fixing index 4
/usr/bin/myisamchk: Disk is full writing '/var/tmp/STxLuDlb' (Errcode: 28). Waiting for someone to free space... (Expect up to 60 secs delay for server to continue after freeing disk space)
/usr/bin/myisamchk: Retry in 60 secs. Message reprinted in 600 secs
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: db repair failed

Post by scottwilkerson »

Code: Select all

/dev/mapper/VolGroup00-LogVol01   13G   13G   20K 100% /var
With /var being 100% full, you are going to need to increase the size of that volume before running the repair as it needs to be able to use some space to perform the repair.
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
rtsupport
Posts: 188
Joined: Mon Jan 08, 2018 4:30 am

Re: db repair failed

Post by rtsupport »

Increasing the space for drive would not be possible here. as it was working fine before so why this creating so much logs now?

or to make a space can we move existing file in different directory? or can we delete/purge them ? and then will start repair.

Please refer last post for files which are lying down on both DIR.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: db repair failed

Post by scottwilkerson »

rtsupport wrote:as it was working fine before so why this creating so much logs now?
The /var path contains all the mysql and postgres data and will grow over time
rtsupport wrote:or to make a space can we move existing file in different directory?
Possibly, but you cannot remove anything from the following directories as they contain the data of your databases

Code: Select all

/var/lib/mysql
/var/lib/pgsql
You may be able to find some log in the following directory that could be truncated.

Code: Select all

/var/log
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
rtsupport
Posts: 188
Joined: Mon Jan 08, 2018 4:30 am

Re: db repair failed

Post by rtsupport »

"The /var path contains all the mysql and postgres data and will grow over time"

replying to this.. on PRD environment we are having /var with same size, but its been more than 6 years its not filled yet, and as this is test VM where we are planning to migrate with latest version so i guess there is some configuration which we can apply to prevent such kind of situation in feature.

Also as this VM data (in terms of logs/graph/history) is not important as of now so keeping this in mind can we move or delete the content from highlighted directory?

But just to let you know we don't want miss any host/service/contact/groups... configuration. so apart from increasing the /var partition space do we have any other better approach which we can follow?

we have already tried to truncated any file from /var/log but nothing is there which we can do.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: db repair failed

Post by scottwilkerson »

Not sure if the system will let you run it without any available disk space but you can try to run the following to remove some of the data from postgres

Code: Select all

echo "truncate table xi_events; truncate table xi_meta; truncate table xi_eventqueue;" | psql nagiosxi nagiosxi
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
rtsupport
Posts: 188
Joined: Mon Jan 08, 2018 4:30 am

Re: db repair failed

Post by rtsupport »

Thank Scott, you always help with your magic :)

after running this command --
DB repaired done.
nagios reconfigure done.
/var back to 63%
all services restarted = observed - that performance grapher service was indicating red on GUI, restarted npcd service from cmd but nothing was done, so restarted from GUI which fix it dose we need to focus anything?

can you please explain little bit about the command we used what that command did? can we use this command in PRD.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: db repair failed

Post by scottwilkerson »

rtsupport wrote:can you please explain little bit about the command we used what that command did? can we use this command in PRD.
This command removed any old event items that are the data that would be used to populate notifications, etc. This is usually only stored for a short period and I think you may have had a bunch of data in there from recent events.

I would still suggest finding a way to be able to resize the /var area as you may bump up against this in the future.
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
rtsupport
Posts: 188
Joined: Mon Jan 08, 2018 4:30 am

Re: db repair failed

Post by rtsupport »

thank you! for your comments any advise for from last chat question?

all services restarted = observed - that performance grapher service was indicating red on GUI, restarted npcd service from cmd but nothing was done, so restarted from GUI which fix it dose we need to focus anything?

Also when we rebooted the server we came in this situation where we faced lots of issue as explained in beginning of the chat so we will plan a reboot in weekend and will see if will face any issue again or we will be good to go.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: db repair failed

Post by scottwilkerson »

rtsupport wrote:all services restarted = observed - that performance grapher service was indicating red on GUI, restarted npcd service from cmd but nothing was done, so restarted from GUI which fix it dose we need to focus anything?
This depends on the type of server (OS and version) and what command you used to restart the npcd service. On CentOS/RHEL 7 you should use

Code: Select all

systemctl restart npcd
As for the reboot, when doing so make sure you are rebooting using the reboot command from the CLI and not just powering down a VM or physical machine.
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
Locked