Modification of current config to include NLS

[GhostRider2110]

According to python documentation, If facility is not specified, LOG_USER is used.. Since in the code I only see that "class" and "address" being specified then I would say it's LOG_USER.

[mcapra]

Well, you could certainly try sending all messages on the user facility to that log file though that may have unintended consequences if other applications are using that facility. The other option would be to change the facility used by the Python code to one of the locals (local0-local7).

Try commenting/removing everything in the current working file and just having this:

Code: Select all

user.* /var/opt/lrms/log/jupiter.log

I would suggest starting with a fresh jupiter.log or using a whole new file to observe what gets sent to the file.

See what sort of behavior that produces. No templates, just having the user facility go to a file.

[GhostRider2110]

Well, you could certainly try sending all messages on the user facility to that log file though that may have unintended consequences if other applications are using that facility. The other option would be to change the facility used by the Python code to one of the locals (local0-local7).

Try commenting/removing everything in the current working file and just having this:

Code: Select all

user.* /var/opt/lrms/log/jupiter.log

I would suggest starting with a fresh jupiter.log or using a whole new file to observe what gets sent to the file.

See what sort of behavior that produces. No templates, just having the user facility go to a file.

Thanks for the reply. I got some replies back from the rsyslog support forum and what I suspected may be happening. Because rsyslog is being use to send to a file, based on

Code: Select all

if ($msg contains "[ jupiter ]")

, then using rsyslog to process the file, I'm getting into a loop as the msg is being processed again.

I'm am going to adjust the code to use one of the local facilities and see if I can isolate and send to the file that way, then I might be able to break the loop...

[GhostRider2110]

I have made some changes. I changed the code to send to local6 facility.

Then I changed the 89-jupiter.conf file to send local6 to /var/opt/lrms/log/jupiter-local6.log

Code: Select all

[root@igaqarep rsyslog.d]# cat 89-jupiter.conf 

$template JupiterFormat,"%TIMESTAMP:::date-rfc3339% %msg:::sp-if-no-1st-sp% %msg:::drop-last-lf%\n"
local6.* /var/opt/lrms/log/jupiter-local6.log;JupiterFormat

I did do this once without the template and got normal logs like:

Mar 29 11:44:03 igaqarep journal: - [ jupiter ] - 0.0001740456 - a22cda22-04c9-45bf-8f17-53ab2d0679fc - site:dev - INFO -- ============================================================ - jupiter.lib.middleware:59
Mar 29 11:44:03 igaqarep journal: - [ jupiter ] - 0.0004138947 - a22cda22-04c9-45bf-8f17-53ab2d0679fc - site:dev - INFO -- Received GET request. - jupiter.lib.middleware:61

Then I added the template back and now I am getting duplicate entries:

2017-03-29T11:44:03.169749-04:00 - [ jupiter ] - 0.0005168915 - a22cda22-04c9-45bf-8f17-53ab2d0679fc - site:dev - INFO -- Path: /dev/repository/version - jupiter.lib.middleware:63
Mar 29 11:44:03 igaqarep journal: - [ jupiter ] - 0.0005168915 - a22cda22-04c9-45bf-8f17-53ab2d0679fc - site:dev - INFO -- Path: /dev/repository/version - jupiter.lib.middleware:63
2017-03-29T11:44:03.170057-04:00 - [ jupiter ] - 0.0006430149 - a22cda22-04c9-45bf-8f17-53ab2d0679fc - site:dev - INFO -- User: admin - jupiter.lib.middleware:65
Mar 29 11:44:03 igaqarep journal: - [ jupiter ] - 0.0006430149 - a22cda22-04c9-45bf-8f17-53ab2d0679fc - site:dev - INFO -- User: admin - jupiter.lib.middleware:65

I think we are getting closer.

[avandemore]

It's very difficult to answer your question without a full context meaning all of your rsyslog configs.

[GhostRider2110]

It's very difficult to answer your question without a full context meaning all of your rsyslog configs.

Complete rsyslog configs should be earlier in the post... If you need more, let me know I can repost....

[mcapra]

Apologies if we missed it, but can we also get a fresh copy of your /etc/rsyslog.conf file? We may need to explicitly exclude local6 from some default processors.

[GhostRider2110]

No problem, thanks. That did lead me to find out why I was getting duplicate entries into the /var/opt/lrms/log/jupiter-local6.log. I had forgot to remove the initial local6 test. Here is the rsyslog.conf file.

Code: Select all

# rsyslog configuration file

# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html

#### MODULES ####

# The imjournal module bellow is now used as a message source instead of imuxsock.
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$SystemLogRateLimitBurst 5000
$ModLoad imjournal # provides access to the systemd journal
$imjournalRatelimitInterval 100
$imjournalRatelimitBurst 60000
#$ModLoad imklog # reads kernel messages (the same are read from journald)
#$ModLoad immark  # provides --MARK-- message capability

# Provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun 514

# Provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun 514


#### GLOBAL DIRECTIVES ####

# Where to place auxiliary files
$WorkDirectory /var/lib/rsyslog

# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on

# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf

# Turn off message reception via local log socket;
# local messages are retrieved through imjournal now.
$OmitLocalLogging on

# File to store the position in the journal
$IMJournalStateFile imjournal.state


#### RULES ####

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                /var/log/messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure

# Log all the mail messages in one place.
mail.*                                                  -/var/log/maillog


# Log cron stuff
cron.*                                                  /var/log/cron

# Everybody gets emergency messages
*.emerg                                                 :omusrmsg:*

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          /var/log/spooler

# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log
#local6.*						/var/opt/lrms/log/jupiter-local6.log

# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList   # run asynchronously
#$ActionResumeRetryCount -1    # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
# ### end of the forwarding rule ###

[mcapra]

Ah, that makes sense. From where you are now, I think using our "Linux Files" log source setup guide (from the NLS GUI) against the /var/opt/lrms/log/jupiter-local6.log file should get those entries into NLS now. I would just be sure that logic is baked into a totally separate rsyslog config file from 89-jupiter.conf given the previous infinite timestamp duplication problems.

[GhostRider2110]

Ah, that makes sense. From where you are now, I think using our "Linux Files" log source setup guide (from the NLS GUI) against the /var/opt/lrms/log/jupiter-local6.log file should get those entries into NLS now. I would just be sure that logic is baked into a totally separate rsyslog config file from 89-jupiter.conf given the previous infinite timestamp duplication problems.

So create a 90-jupiter.conf with the NLS forwarding in it?