I can't create users

[nagioscarnovale]

Hey everyone,

I can't create users via web GUI
From the elasticsearch logs I see the following exception

Seems a problem of the variable "created_by" on index "nagioslogserver"



/var/log/elasticsearch # tail -f 05b4bd73-2fd2-4334-9b72-23d5a6f5dab6.log

GFvFv0TbSXqNzLeQeGB9fg], [P], s[STARTED]: Failed to execute [index {[nagioslogserver][user][AWuUMXWlFgN8zqhZODTV], source[{"name":"usertest","username":"usertest","email":"nicola.carnovale@par-tec.it","password":"3ad361936ed9ef8d835d745adf24d719186e784ec6fe9a525eeb0045883d5f91","apiaccess":"0","apikey":"","auth_type":"user","language":"default","auth_settings":[],"perms":{"alerts":["view"],"configure":[]},"created":"2019-06-26 16:29:57","created_by":"AWkzrZZRvGl2Usk28wcv","default_dashboard":"\/dashboard\/elasticsearch\/AWuUMXWcFgN8zqhZODTU"}]}]
org.elasticsearch.index.mapper.MapperParsingException: failed to parse [created_by]
at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:411)
at org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(ObjectMapper.java:706)
at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:497)
at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:544)
at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:493)
at org.elasticsearch.index.shard.IndexShard.prepareCreate(IndexShard.java:465)
at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:201)
at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$PrimaryPhase.performOnPrimary(TransportShardReplicationOperationAction.java:574)
at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$PrimaryPhase$1.doRun(TransportShardReplicationOperationAction.java:440)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:36)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NumberFormatException: For input string: "AWkzrZZRvGl2Usk28wcv"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:589)
at java.lang.Long.parseLong(Long.java:631)
at org.elasticsearch.common.xcontent.support.AbstractXContentParser.longValue(AbstractXContentParser.java:145)
at org.elasticsearch.index.mapper.core.LongFieldMapper.innerParseCreateField(LongFieldMapper.java:288)
at org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:239)
at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:401)
... 12 more

I saw the segunte post (similar to my problem)
https://support.nagios.com/forum/viewto ... 37&t=45085

and also in my case I have the variable created_by of type long

but I didn't understand how to fix the problem


[root@nagioslogserver-02 indices]# curl -XGET 'http://localhost:9200/nagioslogserver/_mapping/' | grep created_by
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 4093 100 4093 0 0 661k 0 --:--:-- --:--:-- --:--:-- 799k
{"nagioslogserver":{"mappings":{"alert":{"_timestamp":{"enabled":true},"properties":{"active":{"type":"long"},"alert_crit_only":{"type":"long"},"check_interval":{"type":"string"},"created":{"type":"long"},"created_by":{"type":"string"},"critical":{"type":"string"},"dash_query":{"type":"string"},"last_output":{"type":"string"},"last_run":{"type":"long"},"last_status":{"type":"string"},"lookback_period":{"type":"string"},"method":{"type":"object"},"name":{"type":"string"},"next_run":{"type":"long"},"query":{"type":"string"},"warning":{"type":"string"}}},"snapshot":{"_timestamp":{"enabled":true},"properties":{"auto":{"type":"long"},"clean_filename":{"type":"string"},"created":{"type":"long"},"filename":{"type":"string"},"name":{"type":"string"},"path":{"type":"string"}}},"_default_":{"_timestamp":{"enabled":true},"properties":{}},"commands":{"_timestamp":{"enabled":true},"properties":{"active":{"type":"long"},"args":{"properties":{"created":{"type":"long"},"id":{"type":"string"},"sh_created":{"type":"long"},"sh_id":{"type":"string"},"timezone":{"type":"string"}}},"command":{"type":"string"},"created":{"type":"string"},created_by":{"type":"string"},"frequency":{"type":"string"},"last_run_output":{"type":"string"},"last_run_status":{"type":"string"},"last_run_time":{"type":"string"},"node":{"type":"string","index":"not_analyzed"},"run_time":{"type":"long"},"status":{"type":"string"},"type":{"type":"string"}}},"node":{"_timestamp":{"enabled":true},"properties":{"address":{"type":"string","index":"not_analyzed"},"config_filters":{"properties":{"active":{"type":"long"},"name":{"type":"string"},"raw":{"type":"string"}}},"config_inputs":{"properties":{"active":{"type":"long"},"name":{"type":"string"},"raw":{"type":"string"}}},"config_outputs":{"type":"object"},"elasticsearch":{"properties":{"message":{"type":"string"},"pid":{"type":"string"},"status":{"type":"string"}}},"hostname":{"type":"string"},"last_updated":{"type":"long"},"logstash":{"properties":{"message":{"type":"string"},"pid":{"type":"string"},"status":{"type":"string"}}},"ls_release":{"type":"long"},"ls_version":{"type":"string"}}},"snmp_reactor":{"_timestamp":{"enabled":true},"properties":{"address":{"type":"string"},"auth_level":{"type":"string"},"auth_password":{"type":"string"},"auth_protocol":{"type":"string"},"name":{"type":"string"},"port":{"type":"string"},"priv_password":{"type":"string"},"priv_protocol":{"type":"string"},"username":{"type":"string"},"version":{"type":"string"}}},"user":{"_timestamp":{"enabled":true},"properties":{"apiaccess":{"type":"string"},"apikey":{"type":"string"},"auth_settings":{"properties":{"ad_username":{"type":"string"},"auth_server_id":{"type":"string"},"dn":{"type":"string"},"type":{"type":"string"}}},"auth_type":{"type":"string"},"created":{"type":"string"},"created_by":{"type":"long"},"default_dashboard":{"type":"string"},"email":{"type":"string"},"lang":{"type":"string"},"language":{"type":"string"},"name":{"type":"string"},"password":{"type":"string"},"perms":{"properties":{"alerts":{"type":"string"}}},"username":{"type":"string"}}},"cf_option":{"_timestamp":{"enabled":true},"properties":{"created":{"type":"string"},"created_by":{"type":"string"},"modified_by":{"type":"string"},"value":{"type":"string"}}},"query":{"_timestamp":{"enabled":true},"properties":{"active":{"type":"long"},"created_by":{"type":"string"},"created_id":{"type":"string"},"dash_query":{"type":"string"},"imported":{"type":"long"},"name":{"type":"string"},"raw":{"type":"string"},"services":{"type":"string"},"show_everyone":{"type":"long"}}},"nrdp_server":{"_timestamp":{"enabled":true},"properties":{"address":{"type":"string"},"name":{"type":"string"},"token":{"type":"string"}}},"reactor_server":{"_timestamp":{"enabled":true},"properties":{"address":{"type":"string"},"apikey":{"type":"string"},"name":{"type":"string"}}},"email_template":{"_timestamp":{"enabled":true},"properties":{"body":{"type":"string"},"created_by":{"type":"string"},"last_edit":{"type":"string"},"modified_by":{"type":"string"},"name":{"type":"string"},"subject":{"type":"string"}}}}}}
[root@nagioslogserver-02 indices]#


Thanks in advance
Nick

[cdienger]

Please provide PM me a profile from the system. It can be gathered under Admin > System > System Status > Download System Profile or from the command line with:

/usr/local/nagioslogserver/scripts/profile.sh

This will create /tmp/system-profile.tar.gz.

Note that this file can be very large and may not be able to be uploaded through the system. This is usually due to the logs in the logstash and/or elasticseach directories found in it. If it is too large, please open the profile, extract these directories/files and send them separately.

[nagioscarnovale]

Thank you very much for your reply
cdienger

Attached there are 2 files "system-profile.tar.gz"
coming from the 2 nodes NLS

Thanks in advance
Nick

[nagioscarnovale]

attached the 2° file

[cdienger]

There appears to be a problem with the systems running out of space and being unable to allocate enough space to share shards. What do your settings look like under Admin > System > Snapshots & Maintenance? Are you saving snapshots to a remote repo? What is the "Delete indexes older than" field set to? By default it is 0 which will cause the NLS machine to hold on to data indefinitely. You'll want to set this to delete older data from the NLS server to keep the drive from filling.

[nagioscarnovale]

These days we have installed the version directly
"Nagios Log Server 2.0.8" and the problem is solved

Thanks so much for the support

[cdienger]

Glad to hear!