Incomplete Logs

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
ysingh
Posts: 5
Joined: Wed Mar 13, 2019 12:02 pm

Incomplete Logs

Post by ysingh »

I am not able to see complete logs of one API call on NLS. For API call some are logs are visible on NLS from some are not from same instance.

Regards,
Yaduraj Singh
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Incomplete Logs

Post by scottwilkerson »

Can you give an example of what you are doing and the results you are getting?
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
ysingh
Posts: 5
Joined: Wed Mar 13, 2019 12:02 pm

Re: Incomplete Logs

Post by ysingh »

Let's assume, in a API call, 10 steps are executing. Sometimes I only get last 5 steps in NLS. First 5 are missing.

Regards,
User avatar
cdienger
Support Tech
Posts: 5045
Joined: Tue Feb 07, 2017 11:26 am

Re: Incomplete Logs

Post by cdienger »

What API calls are you making? Can you provide some screenshots so we can understand the problem better?
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
ysingh
Posts: 5
Joined: Wed Mar 13, 2019 12:02 pm

Re: Incomplete Logs

Post by ysingh »

On server we are getting below logs.

info: finicityRequestId: FIN292309034
info: processMessageAndReply started for request id FIN292309034: {"requestType":"XXXXXXX","partnerId":"XXXXXXX","customerId":"XXXXXXX","institutionId":"XXXXXXX","keys":{"PIN":"*****","ID":"*****"}}
FIN292309034 browserConfig { args:
2019-03-25T13:00:26.406Z - info: FinicityRequestId: FIN292309034 Institution: XXXXXXX, Customer: XXXXXXX, RequestType: discovery, - Request timeout type: normal, value: 600000.
2019-03-25T13:00:28.816Z - info: FinicityRequestId: FIN292309034 Institution: XXXXXXX, Customer: XXXXXXX, RequestType: discovery, - Executed on IP address = XXXXXXX:XXXX
2019-03-25T13:00:28.816Z - info: FinicityRequestId: FIN292309034 Institution: XXXXXXX, Customer: XXXXXXX, RequestType: discovery, - Started script Login
2019-03-25T13:00:35.322Z - info: FinicityRequestId: FIN292309034 Institution: XXXXXXX, Customer: XXXXXXX, RequestType: discovery, - Accounts Page Reached
2019-03-25T13:00:35.322Z - info: FinicityRequestId: FIN292309034 Institution: XXXXXXX, Customer: XXXXXXX, RequestType: discovery, - Completed script login, success: true, duration: 6506
2019-03-25T13:00:35.323Z - info: FinicityRequestId: FIN292309034 Institution: XXXXXXX, Customer: XXXXXXX, RequestType: discovery, - started discovery
2019-03-25T13:00:37.425Z - info: FinicityRequestId: FIN292309034 Institution: XXXXXXX, Customer: XXXXXXX, RequestType: discovery, - clearSessionTimeout: timer cleared.
2019-03-25T13:00:37.426Z - warn: FinicityRequestId: FIN292309034 Institution: XXXXXXX, Customer: XXXXXXX, RequestType: discovery, - Called clear session timeout for nonexisting timeout
info: processMessageAndReply completed for Request id FIN292309034 Institution Id: XXXXXXX, Partner Id: XXXXXXX. Duration=11373 Request: {"requestType":"DISCOVERY","partnerId":"XXXXXXX","customerId":"XXXXXXX","institutionId":"XXXXXXX","keys":{"PIN":"*****","ID":"*****"}}

But in NLS we are getting less logs. Please find the attachment.
You do not have the required permissions to view the files attached to this post.
User avatar
cdienger
Support Tech
Posts: 5045
Joined: Tue Feb 07, 2017 11:26 am

Re: Incomplete Logs

Post by cdienger »

Looks like a problem with parsing. Can you PM me a copy of the log as well as a copy of the logstash config? The logstash config can be gathered by going to Configure > Global (All Instances) > Global Config > View > All Files Combined.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
User avatar
mcapra
Posts: 3739
Joined: Thu May 05, 2016 3:54 pm

Re: Incomplete Logs

Post by mcapra »

Which input are these going to? How did you configure the logging agent on the remote machine?

If it's going to a Logstash syslog input, these logs:

Code: Select all

info: finicityRequestId: FIN292309034
info: processMessageAndReply started for request id FIN292309034: {"requestType":"XXXXXXX","partnerId":"XXXXXXX","customerId":"XXXXXXX","institutionId":"XXXXXXX","keys":{"PIN":"*****","ID":"*****"}}
FIN292309034 browserConfig { args:
2019-03-25T13:00:26.406Z - info: FinicityRequestId: FIN292309034 Institution: XXXXXXX, Customer: XXXXXXX, RequestType: discovery, - Request timeout type: normal, value: 600000.
2019-03-25T13:00:28.816Z - info: FinicityRequestId: FIN292309034 Institution: XXXXXXX, Customer: XXXXXXX, RequestType: discovery, - Executed on IP address = XXXXXXX:XXXX
2019-03-25T13:00:28.816Z - info: FinicityRequestId: FIN292309034 Institution: XXXXXXX, Customer: XXXXXXX, RequestType: discovery, - Started script Login
2019-03-25T13:00:35.322Z - info: FinicityRequestId: FIN292309034 Institution: XXXXXXX, Customer: XXXXXXX, RequestType: discovery, - Accounts Page Reached
2019-03-25T13:00:35.322Z - info: FinicityRequestId: FIN292309034 Institution: XXXXXXX, Customer: XXXXXXX, RequestType: discovery, - Completed script login, success: true, duration: 6506
2019-03-25T13:00:35.323Z - info: FinicityRequestId: FIN292309034 Institution: XXXXXXX, Customer: XXXXXXX, RequestType: discovery, - started discovery
2019-03-25T13:00:37.425Z - info: FinicityRequestId: FIN292309034 Institution: XXXXXXX, Customer: XXXXXXX, RequestType: discovery, - clearSessionTimeout: timer cleared.
2019-03-25T13:00:37.426Z - warn: FinicityRequestId: FIN292309034 Institution: XXXXXXX, Customer: XXXXXXX, RequestType: discovery, - Called clear session timeout for nonexisting timeout
info: processMessageAndReply completed for Request id FIN292309034 Institution Id: XXXXXXX, Partner Id: XXXXXXX. Duration=11373 Request: {"requestType":"DISCOVERY","partnerId":"XXXXXXX","customerId":"XXXXXXX","institutionId":"XXXXXXX","keys":{"PIN":"*****","ID":"*****"}} 
Are not RFC-3164/RFC-5424 compliant and will cause issues in the parsing stages. The standard expects every line to have, among other things, a timestamp.

I would suggest shipping these logs to some other sort of input rule. Perhaps the "Import Files - Raw" one that ships with Nagios Log Server by default. The change could be as simple as altering the port your logging agent on the remote machine (rsyslog, syslog-ng, nxlog, fluentd, etc) is using.
Former Nagios employee
https://www.mcapra.com/
User avatar
cdienger
Support Tech
Posts: 5045
Joined: Tue Feb 07, 2017 11:26 am

Re: Incomplete Logs

Post by cdienger »

Thanks for the input, @mcapra!
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
ysingh
Posts: 5
Joined: Wed Mar 13, 2019 12:02 pm

Re: Incomplete Logs

Post by ysingh »

@mcapra @cdienger As I mentioned in the starting, sometimes I get the full logs but some times not.
User avatar
cdienger
Support Tech
Posts: 5045
Joined: Tue Feb 07, 2017 11:26 am

Re: Incomplete Logs

Post by cdienger »

@@ysingh, as mentioned in our previous responses, there appears to be an issue with the logs not having the proper formatting. Try configuring the logging agent to send logs to port 2056 instead of 5544.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
Locked