Settings on the Backup and Maintenance are not staying

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
Locked
kseasolutions
Posts: 6
Joined: Mon Jun 19, 2017 10:20 am

Settings on the Backup and Maintenance are not staying

Post by kseasolutions »

When I am making changes to the settings on the Backup/Maintenance interface on Nagios Logging server 1.4.4 When I click Save it does not stay persistent. It goes back to the default settings. In addition bloom filter cache is missing from the user interface as well.

Any help would be great.
Top
User avatar
cdienger
Support Tech
Posts: 5045
Joined: Tue Feb 07, 2017 11:26 am

Re: Settings on the Backup and Maintenance are not staying

Post by cdienger »

Sounds like a possible issue with the nagioslogserver index. Please run the following and post the output:

Code: Select all

curl -XGET 'http://localhost:9200/_cluster/health/*?level=shards&pretty'
I'd also like you run:

Code: Select all

tail -f /var/log/elasticsearch/<CLUSTER_ID>.log
Where the CLUSTER_ID can be found by looking at the second line of output from the previous command. Let this run while you try to write and save the changes again in the web UI. Provide anything that may be generated on the cli when you do.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
Top
kseasolutions
Posts: 6
Joined: Mon Jun 19, 2017 10:20 am

Re: Settings on the Backup and Maintenance are not staying

Post by kseasolutions »

curl -XGET 'http://localhost:9200/_cluster/health/* ... rds&pretty'

Code: Select all

{
  "cluster_name" : "redacted----",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 25,
  "active_shards" : 25,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 25,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "indices" : {
    "nagioslogserver_log" : {
      "status" : "yellow",
      "number_of_shards" : 5,
      "number_of_replicas" : 1,
      "active_primary_shards" : 5,
      "active_shards" : 5,
      "relocating_shards" : 0,
      "initializing_shards" : 0,
      "unassigned_shards" : 5,
      "shards" : {
        "0" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        },
        "1" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        },
        "2" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        },
        "3" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        },
        "4" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        }
      }
    },
    "nagioslogserver" : {
      "status" : "yellow",
      "number_of_shards" : 5,
      "number_of_replicas" : 1,
      "active_primary_shards" : 5,
      "active_shards" : 5,
      "relocating_shards" : 0,
      "initializing_shards" : 0,
      "unassigned_shards" : 5,
      "shards" : {
        "0" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        },
        "1" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        },
        "2" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        },
        "3" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        },
        "4" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        }
      }
    },
    "kibana-int" : {
      "status" : "yellow",
      "number_of_shards" : 5,
      "number_of_replicas" : 1,
      "active_primary_shards" : 5,
      "active_shards" : 5,
      "relocating_shards" : 0,
      "initializing_shards" : 0,
      "unassigned_shards" : 5,
      "shards" : {
        "0" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        },
        "1" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        },
        "2" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        },
        "3" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        },
        "4" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        }
      }
    },
    "logstash-2017.08.08" : {
      "status" : "yellow",
      "number_of_shards" : 5,
      "number_of_replicas" : 1,
      "active_primary_shards" : 5,
      "active_shards" : 5,
      "relocating_shards" : 0,
      "initializing_shards" : 0,
      "unassigned_shards" : 5,
      "shards" : {
        "0" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        },
        "1" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        },
        "2" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        },
        "3" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        },
        "4" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        }
      }
    },
    "logstash-2017.08.09" : {
      "status" : "yellow",
      "number_of_shards" : 5,
      "number_of_replicas" : 1,
      "active_primary_shards" : 5,
      "active_shards" : 5,
      "relocating_shards" : 0,
      "initializing_shards" : 0,
      "unassigned_shards" : 5,
      "shards" : {
        "0" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        },
        "1" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        },
        "2" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        },
        "3" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        },
        "4" : {
          "status" : "yellow",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 1
        }
      }
    }
  }
}
Last edited by kseasolutions on Thu Aug 10, 2017 12:35 pm, edited 1 time in total.
Top
kseasolutions
Posts: 6
Joined: Mon Jun 19, 2017 10:20 am

Re: Settings on the Backup and Maintenance are not staying

Post by kseasolutions »

From the tail I can see the error fairly clear

Code: Select all

[2017-08-09 15:45:18,741][DEBUG][action.index             ] [--redact--] [nagioslogserver][1], node[--redact--], [P], s[STARTED]: [color=#FF0000]Failed to execute [index {[nagioslogserver][cf_option][maintenance_settings], source[{"created":"2017-01-03 22:55:13","created_by":0,"value":"a:6:{s:13:\"optimize_time\";s:1:\"5\";s:10:\"close_time\";s:1:\"7\";s:11:\"delete_time\";s:2:\"30\";s:10:\"repository\";s:15:\"Archive_indexes\";s:20:\"delete_snapshot_time\";s:2:\"90\";s:6:\"active\";s:1:\"1\";}","modified_by":"AVz2hp3mHpAy0Akc-9b_"}]}][/color]
org.elasticsearch.index.mapper.MapperParsingException: failed to parse [modified_by]
        at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:411)
        at org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(ObjectMapper.java:706)
        at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:497)
        at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:544)
        at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:493)
        at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:493)
        at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:192)
        at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$PrimaryPhase.performOnPrimary(TransportShardReplicationOperationAction.java:574)
        at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$PrimaryPhase.routeRequestOrPerformLocally(TransportShardReplicationOperationAction.java:444)
        at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$PrimaryPhase.doRun(TransportShardReplicationOperationAction.java:370)
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:36)
        at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction.doExecute(TransportShardReplicationOperationAction.java:112)
        at org.elasticsearch.action.index.TransportIndexAction.innerExecute(TransportIndexAction.java:136)
        at org.elasticsearch.action.index.TransportIndexAction.doExecute(TransportIndexAction.java:114)
        at org.elasticsearch.action.index.TransportIndexAction.doExecute(TransportIndexAction.java:63)
        at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:75)
        at org.elasticsearch.action.update.TransportUpdateAction.shardOperation(TransportUpdateAction.java:217)
        at org.elasticsearch.action.update.TransportUpdateAction.shardOperation(TransportUpdateAction.java:170)
        at org.elasticsearch.action.support.single.instance.TransportInstanceSingleOperationAction$AsyncSingleAction$1.run(TransportInstanceSingleOperationAction.java:187)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NumberFormatException: For input string: "AVz2hp3mHpAy0Akc-9b_"
        at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
        at java.lang.Long.parseLong(Long.java:441)
        at java.lang.Long.parseLong(Long.java:483)
        at org.elasticsearch.common.xcontent.support.AbstractXContentParser.longValue(AbstractXContentParser.java:145)
        at org.elasticsearch.index.mapper.core.LongFieldMapper.innerParseCreateField(LongFieldMapper.java:288)
        at org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:239)
        at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:401)
        ... 21 more
Last edited by kseasolutions on Thu Aug 10, 2017 12:37 pm, edited 1 time in total.
Top
User avatar
mcapra
Posts: 3739
Joined: Thu May 05, 2016 3:54 pm

Re: Settings on the Backup and Maintenance are not staying

Post by mcapra »

I'm betting modified_by and created_by are both typed as longs rather than strings. I believe this is a known issue within NLS, though I don't have a reference number for the bug.

Can you share the full output of the following command executed from the CLI of one of your Nagios Log Server machines:

Code: Select all

curl -XGET 'http://localhost:9200/nagioslogserver/_mapping/'
Please wrap it in

Code: Select all

 tags as the output will be quite long.

Also, which account are you making these changes from? I'm betting the default nagiosadmin account is able to make the changes without issue.
Former Nagios employee
https://www.mcapra.com/
Top
kseasolutions
Posts: 6
Joined: Mon Jun 19, 2017 10:20 am

Re: Settings on the Backup and Maintenance are not staying

Post by kseasolutions »

curl -XGET 'http://localhost:9200/nagioslogserver/_mapping/'

Code: Select all

{"nagioslogserver":{"mappings":{"user":{"properties":{"apiaccess":{"type":"string"},"apikey":{"type":"string"},"auth_settings":{"properties":{"ad_username":{"type":"string"},"auth_server_id":{"type":"string"},"dn":{"type":"string"},"type":{"type":"string"}}},"auth_type":{"type":"string"},"created":{"type":"string"},"created_by":{"type":"long"},"default_dashboard":{"type":"string"},"email":{"type":"string"},"lang":{"type":"string"},"language":{"type":"string"},"name":{"type":"string"},"password":{"type":"string"},"username":{"type":"string"}}},"node":{"properties":{"address":{"type":"string"},"config_filters":{"properties":{"active":{"type":"long"},"name":{"type":"string"},"raw":{"type":"string"}}},"config_inputs":{"properties":{"active":{"type":"long"},"name":{"type":"string"},"raw":{"type":"string"}}},"elasticsearch":{"properties":{"message":{"type":"string"},"pid":{"type":"string"},"status":{"type":"string"}}},"hostname":{"type":"string"},"last_updated":{"type":"long"},"logstash":{"properties":{"message":{"type":"string"},"pid":{"type":"string"},"status":{"type":"string"}}},"ls_release":{"type":"long"},"ls_version":{"type":"string"}}},"snapshot":{"properties":{"auto":{"type":"long"},"clean_filename":{"type":"string"},"created":{"type":"long"},"filename":{"type":"string"},"name":{"type":"string"},"path":{"type":"string"}}},"cf_option":{"properties":{"created":{"type":"string"},"created_by":{"type":"long"},"modified_by":{"type":"long"},"value":{"type":"string"}}},"commands":{"properties":{"active":{"type":"long"},"args":{"properties":{"path":{"type":"string"},"sh_created":{"type":"long"},"sh_id":{"type":"string"},"timezone":{"type":"string"}}},"command":{"type":"string"},"created":{"type":"string"},"created_by":{"type":"string"},"frequency":{"type":"string"},"last_run_output":{"type":"string"},"last_run_status":{"type":"string"},"last_run_time":{"type":"string"},"node":{"type":"string"},"run_time":{"type":"long"},"status":{"type":"string"},"type":{"type":"string"}}},"query":{"properties":{"created_by":{"type":"string"},"created_id":{"type":"string"},"edit_type":{"type":"string"},"imported":{"type":"long"},"name":{"type":"string"},"raw":{"type":"string"},"services":{"type":"string"},"show_everyone":{"type":"long"},"title":{"type":"string"}}}}}}
Top
User avatar
mcapra
Posts: 3739
Joined: Thu May 05, 2016 3:54 pm

Re: Settings on the Backup and Maintenance are not staying

Post by mcapra »

Yup:

Code: Select all

"cf_option": {
				"properties": {
					"created": {
						"type": "string"
					},
					"created_by": {
						"type": "long"
					},
					"modified_by": {
						"type": "long"
					},
					"value": {
						"type": "string"
					}
				}
			}
Have you tried making the changes by using the nagiosadmin user? I'll defer to a tech in terms of a fix, since I don't recall specifically what the "official" fix was.
Former Nagios employee
https://www.mcapra.com/
Top
kseasolutions
Posts: 6
Joined: Mon Jun 19, 2017 10:20 am

Re: Settings on the Backup and Maintenance are not staying

Post by kseasolutions »

I just made the changes as the official Nagiosadmin user and they stuck this time (though bloom is still not an option)

Previously, I was in an admin account, but not nagiosadmin.
Top
kseasolutions
Posts: 6
Joined: Mon Jun 19, 2017 10:20 am

Re: Settings on the Backup and Maintenance are not staying

Post by kseasolutions »

Thank you for the post @mcapra. I will wait to see what the technical fix is on this.

Thank you once again.
Top
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:
Contact scottwilkerson

Re: Settings on the Backup and Maintenance are not staying

Post by scottwilkerson »

The bloom filter has been removed in the current version as it is no longer used.

This is a remnant of a previous bug, to fix the mapping problem you will want to do the following

create a new system backup

Code: Select all

/usr/local/nagioslogserver/scripts/create_backup.sh
The restore this backup
find the newest file file

Code: Select all

ls -l /store/backups/nagioslogserver/
use that to restore the backup

Code: Select all

/usr/local/nagioslogserver/scripts/restore_backup.sh /store/backups/nagioslogserver/nagioslogserver.2017-08-10.XXXXXXX.tar.gz
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
Top
Locked

Return to “Nagios Log Server”