Log Server creating huge local log file - IIS

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
Locked
rferebee
Posts: 733
Joined: Wed Jul 11, 2018 11:37 am

Log Server creating huge local log file - IIS

Post by rferebee »

Hello,

I've been made aware of an issue with Log Server that is only happening on one of my Windows servers that we are tracking IIS on. It's a Skype Edge server.

We are using the latest nxlog client. As soon as we enable the nxlog service, it starts writing lines into the local log file like this: 2019-03-04 09:10:21 ERROR if-else failed at line 73, character 470 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; procedure 'parse_csv' failed at line 73, character 231 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; Not enough fields in CSV input, expected 22, got 15 in input '2019-03-04 17:10:14 10.131.11.17 POST /groupexpansion/service.svc/WebTicket_Bearer - 443 - 10.231.51.89 OC/16.0.4810.1000+(Skype+for+Business) - 500 0 0 28'

Millions and millions of lines. A 200GB log file had to be deleted 3 separate times over the weekend. I have attached our .CONF file for review. Is there anything you can see that we need to change? This just started occurring and we have not made any changes to our .CONF file.

Thank you.
You do not have the required permissions to view the files attached to this post.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Log Server creating huge local log file - IIS

Post by scottwilkerson »

Did someone change the format of what is logged in your IIS logs?

Per the error, the w3c extension in the nxlog configuration is expecting 23 fields and only seeing 15 in the logs.

this is what the expected fields are:

Code: Select all

<Extension w3c>
    Module             xm_csv
    Fields             $date, $time, $sitename, $computername, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $cs-version, $csUser-Agent, $cs-cookie, $cs-Referer, $cs_host, $sc-status, $sc-substatus, $sc-win32-status, $sc-bytes, $cs-bytes, $time-taken
    FieldTypes         string, string, string, string, string, string, string, string, integer, string, string, string, string, string, string, string, integer, integer, integer, integer, integer, integer
    Delimiter         ' '
    QuoteChar         '"'
    EscapeControl     FALSE
    UndefValue         -
</Extension>
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
rferebee
Posts: 733
Joined: Wed Jul 11, 2018 11:37 am

Re: Log Server creating huge local log file - IIS

Post by rferebee »

No changes have been made to IIS since we started logging a couple months ago. Also, we've been using the same .CONF file since the beginning.

I will however, confirm your reply. The log was having a meltdown because it was expecting 22 fields and only getting 15. After making a manual adjustment to the .CONF to match the specific output of the IIS log files everything started working properly. Each .W3C file has the specific output fields listed at the top of the file the same way it's listed in the .CONF, they need to match explicitly it would seem.

What concerns me is that this behavior started for no reason over the weekend an nearly caused our Skype front end server to crash, even though we've been logging IIS for months. I confirmed with my Sys Admins that no change was made to the IIS configuration prior to this issue occurring.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Log Server creating huge local log file - IIS

Post by scottwilkerson »

I cannot explain what could have changed. Especially since the <Extension w3c> section isn't something that is placed in the config by default.

I am glad to hear it is working.
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
rferebee
Posts: 733
Joined: Wed Jul 11, 2018 11:37 am

Re: Log Server creating huge local log file - IIS

Post by rferebee »

This can be locked. We'll just be sure that our internal IIS configuration is uniform across our environment.

Thank you.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Log Server creating huge local log file - IIS

Post by scottwilkerson »

rferebee wrote:This can be locked. We'll just be sure that our internal IIS configuration is uniform across our environment.

Thank you.
Locking thread
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
Locked