Modification of current config to include NLS

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
GhostRider2110
Posts: 193
Joined: Thu Oct 30, 2014 8:04 am
Location: Indiana
Contact:

Re: Modification of current config to include NLS

Post by GhostRider2110 »

Using that .conf file, the file logging is sending something , but does not seem to be sending to NLS. Also the log to the file is not right. Here is what is being put into the file /var/opt/lrms/log/jupiter.log

Code: Select all

2017-03-27T10:05:32.994701-04:00   2017-03-27T10:05:22.984191-04:00   2017-03-27T10:05:12.973781-04:00   2017-03-27T10:05:02.963226-04:00   2017-03-27T10:04:52.952779-04:00   2017-03-27T10:04:42.942258-04:00   2017-03-27T10:04:32.931596-04:00   2017-03-27T10:04:22.920978-04:00   2017-03-27T10:04:12.910529-04:00   2017-03-27T10:04:02.900056-04:00   2017-03-27T10:03:52.889569-04:00   2017-03-27T10:03:42.879206-04:00   2017-03-27T10:03:32.868793-04:00   2017-03-27T10:03:23.931169-04:00   - [ jupiter ] - 0.0007948875 - 19479a94-0437-4bc0-990b-33dbe1783a2e - site:dev -     INFO -- IP: 10.100.52.117 - jupiter.lib.middleware:67
2017-03-27T10:05:32.994712-04:00   2017-03-27T10:05:22.984201-04:00   2017-03-27T10:05:12.973794-04:00   2017-03-27T10:05:02.963234-04:00   2017-03-27T10:04:52.952787-04:00   2017-03-27T10:04:42.942274-04:00   2017-03-27T10:04:32.931612-04:00   2017-03-27T10:04:22.920986-04:00   2017-03-27T10:04:12.910535-04:00   2017-03-27T10:04:02.900063-04:00   2017-03-27T10:03:52.889576-04:00   2017-03-27T10:03:42.879210-04:00   2017-03-27T10:03:32.868797-04:00   2017-03-27T10:03:23.931395-04:00   - [ jupiter ] - 0.0008809566 - 19479a94-0437-4bc0-990b-33dbe1783a2e - site:dev -     INFO -- User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.110 Safari/537.36 - jupiter.lib.middleware:69
2017-03-27T10:05:32.994721-04:00   2017-03-27T10:05:22.984209-04:00   2017-03-27T10:05:12.973803-04:00   2017-03-27T10:05:02.963244-04:00   2017-03-27T10:04:52.952794-04:00   2017-03-27T10:04:42.942288-04:00   2017-03-27T10:04:32.931625-04:00   2017-03-27T10:04:22.920992-04:00   2017-03-27T10:04:12.910541-04:00   2017-03-27T10:04:02.900069-04:00   2017-03-27T10:03:52.889581-04:00   2017-03-27T10:03:42.879215-04:00   2017-03-27T10:03:32.868801-04:00   2017-03-27T10:03:23.931630-04:00   - [ jupiter ] - 0.0009779930 - 19479a94-0437-4bc0-990b-33dbe1783a2e - site:dev -     INFO -- Parameters: <QueryDict: {}> - jupiter.lib.middleware:71
2017-03-27T10:05:32.994731-04:00   2017-03-27T10:05:22.984218-04:00   2017-03-27T10:05:12.973811-04:00   2017-03-27T10:05:02.963252-04:00   2017-03-27T10:04:52.952805-04:00   2017-03-27T10:04:42.942304-04:00   2017-03-27T10:04:32.931640-04:00   2017-03-27T10:04:22.920998-04:00   2017-03-27T10:04:12.910546-04:00   2017-03-27T10:04:02.900074-04:00   2017-03-27T10:03:52.889586-04:00   2017-03-27T10:03:42.879221-04:00   2017-03-27T10:03:32.868804-04:00   2017-03-27T10:03:23.931869-04:00   - [ jupiter ] - 0.0010619164 - 19479a94-0437-4bc0-990b-33dbe1783a2e - site:dev -     INFO -- ------------------------------------------------------------ - jupiter.lib.middleware:72
2017-03-27T10:05:32.994740-04:00   2017-03-27T10:05:22.984227-04:00   2017-03-27T10:05:12.973819-04:00   2017-03-27T10:05:02.963260-04:00   2017-03-27T10:04:52.952813-04:00   2017-03-27T10:04:42.942321-04:00   2017-03-27T10:04:32.931653-04:00   2017-03-27T10:04:22.921004-04:00   2017-03-27T10:04:12.910552-04:00   2017-03-27T10:04:02.900079-04:00   2017-03-27T10:03:52.889591-04:00   2017-03-27T10:03:42.879225-04:00   2017-03-27T10:03:32.868811-04:00   2017-03-27T10:03:23.936198-04:00   - [ jupiter ] - 0.0064399242 - 19479a94-0437-4bc0-990b-33dbe1783a2e - site:dev -     INFO -- Finished processing request - jupiter.lib.middleware:75
2017-03-27T10:05:32.994750-04:00   2017-03-27T10:05:22.984236-04:00   2017-03-27T10:05:12.973828-04:00   2017-03-27T10:05:02.963269-04:00   2017-03-27T10:04:52.952821-04:00   2017-03-27T10:04:42.942338-04:00   2017-03-27T10:04:32.931668-04:00   2017-03-27T10:04:22.921011-04:00   2017-03-27T10:04:12.910558-04:00   2017-03-27T10:04:02.900084-04:00   2017-03-27T10:03:52.889596-04:00   2017-03-27T10:03:42.879230-04:00   2017-03-27T10:03:32.868869-04:00   10.100.52.117 - - [27/Mar/2017:10:03:15 -0400] "GET /dev/admin/jupiter/lrms_revision/1948166/ HTTP/1.1" 200 1036146 "http://igaqarep/dev/admin/jupiter/lrms_revision/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.110 Safari/537.36"
2017-03-27T10:05:32.994760-04:00   2017-03-27T10:05:22.984246-04:00   2017-03-27T10:05:12.973838-04:00   2017-03-27T10:05:02.963278-04:00   2017-03-27T10:04:52.952830-04:00   2017-03-27T10:04:42.942355-04:00   2017-03-27T10:04:32.931686-04:00   2017-03-27T10:04:22.921033-04:00   2017-03-27T10:04:12.910564-04:00   2017-03-27T10:04:02.900091-04:00   2017-03-27T10:03:52.889601-04:00   2017-03-27T10:03:42.879234-04:00   2017-03-27T10:03:32.868874-04:00   10.100.52.117 - - [27/Mar/2017:10:03:23 -0400] "GET /dev/admin/jsi18n/ HTTP/1.1" 200 2528 "http://igaqarep/dev/admin/jupiter/lrms_revision/1948166/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.110 Safari/537.36"
Looks like multiple time stamps being send to each entry.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Modification of current config to include NLS

Post by scottwilkerson »

mcapra may have better tests on his dev machine but I think these 2 lines at the top shouldn't be there, and you may want to try commenting them out.

Code: Select all

if ($msg contains "jupiter") then /var/opt/lrms/log/jupiter.log;JupiterFormat
if ($msg contains "jupiter") then ~
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
GhostRider2110
Posts: 193
Joined: Thu Oct 30, 2014 8:04 am
Location: Indiana
Contact:

Re: Modification of current config to include NLS

Post by GhostRider2110 »

The only problem with those removed, would that now no longer log to the file /var/opt/lrms/log/jupiter.log.

Those were in a separate file. I was trying to set things up using one file which would send to /var/opt/lrms/log/jupiter.log, as the app sets up on install, and have them sent to the NLS.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Modification of current config to include NLS

Post by scottwilkerson »

Well this line discards anything further

Code: Select all

if ($msg contains "jupiter") then ~
so anything below it will be ignored, I don't believe anything below will be processed
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
GhostRider2110
Posts: 193
Joined: Thu Oct 30, 2014 8:04 am
Location: Indiana
Contact:

Re: Modification of current config to include NLS

Post by GhostRider2110 »

That's what I thought, but I figured you guys were the professionals, and since I have not spent the night at a holiday Inn Express lately, you guys would know better...... :lol:

I'll try removing that line and see what happens...
GhostRider2110
Posts: 193
Joined: Thu Oct 30, 2014 8:04 am
Location: Indiana
Contact:

Re: Modification of current config to include NLS

Post by GhostRider2110 »

Ok, that did get the logs flowing into the NLS, but still getting the multiple timestamps into both the file log and NLS.. Here is an example...

Attached is example of the log.
You do not have the required permissions to view the files attached to this post.
avandemore
Posts: 1597
Joined: Tue Sep 27, 2016 4:57 pm

Re: Modification of current config to include NLS

Post by avandemore »

Going back to @ssax's answer: https://support.nagios.com/forum/viewto ... 99#p140198

That is correctly logging timestamp for me:

Code: Select all

echo "Mar 27 13:01:11 avandemore-centos7 test: testserver testprog: will smith" > /root/jupiter.log
# cat /var/log/jupiter.log
2017-03-27T13:02:49.197176-05:00   testserver testprog: will smith
Previous Nagios employee
GhostRider2110
Posts: 193
Joined: Thu Oct 30, 2014 8:04 am
Location: Indiana
Contact:

Re: Modification of current config to include NLS

Post by GhostRider2110 »

Bummer for me... LOL....

If I remove the forwarding to NLS, the all works fine being sent only to the file.

Code: Select all

[root@igaqarep rsyslog.d]# cat 89-jupiter.conf 
$template JupiterFormat,"%TIMESTAMP:::date-rfc3339% %msg:::sp-if-no-1st-sp% %msg:::drop-last-lf%\n"
if ($msg contains "jupiter") then /var/opt/lrms/log/jupiter.log;JupiterFormat
if ($msg contains "jupiter") then ~

#$ModLoad imfile
#$InputFilePollInterval 10
#$PrivDropToGroup adm
#$WorkDirectory /var/lib/rsyslog

## Input for import_json
#$InputFileName /var/opt/lrms/log/jupiter.log
#$InputFileTag jupiter:
#$InputFileStateFile nls-state-var_opt_lrms_log_jupiter_log # Must be unique for each file being polled
## Uncomment the folowing line to override the default severity for messages
## from this file.
##$InputFileSeverity info
#$InputFilePersistStateInterval 20000
#$InputRunFileMonitor

## Forward to Nagios Log Server and then discard, otherwise these messages
## will end up in the syslog file (/var/log/messages) unless there are other
## overriding rules.
#if $programname == "jupiter" then @@iganagioslog:5583
#if $programname == "jupiter" then ~
So it must be something going on with how rsyslog is processing having the app log to the file, then trying to pick up the file and send to NLS.

I'm kinda stumped....
GhostRider2110
Posts: 193
Joined: Thu Oct 30, 2014 8:04 am
Location: Indiana
Contact:

Re: Modification of current config to include NLS

Post by GhostRider2110 »

Running a little test, I noticed that if I clear out the log, restart rsyslog, all works well.

Attached the /var/opt/lrms/log/jupiter.log. I have put notes in it where I started the log from blank, the where I restarted rsyslog. Seems at the restart of rsyslog, something is putting duplicate entries. The entries seem to be repeat of what is already in the file and adding time stamps each time....
You do not have the required permissions to view the files attached to this post.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Modification of current config to include NLS

Post by scottwilkerson »

GhostRider2110 wrote:Running a little test, I noticed that if I clear out the log, restart rsyslog, all works well.

Attached the /var/opt/lrms/log/jupiter.log. I have put notes in it where I started the log from blank, the where I restarted rsyslog. Seems at the restart of rsyslog, something is putting duplicate entries. The entries seem to be repeat of what is already in the file and adding time stamps each time....
This was why I said you didn't need those 2 lines at the top, because it is going to re-add what it is reading to the same file.

And yes, you would need to restart syslog for the changes to take affect.
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
Locked