Hey everyone,
For background I have NLS but have only used email alerting and nothing too crazy for dashboards. I have a new request that I was trying to figure out but all my ideas don't work. Here is the scenario:
Database system with a web frontend.
When anything is done it is logged and those logs are sent to NLS.
The logs rotate constantly, a new log for every case being worked on(log filename gets date/time added onto it)
When an error occurs NLS sees it and send email to the application owners
Here is the data logged into NLS:
As you can see the log filename is included so I did email alerts, included the last 10 instances and send the log over. The problem is this, the application owners don't have OS access so they can not go grab that file and have to bug the OS admins to go get it, nor do they have access into NLS. They need the file as it has the case # they need to investigate in the log. I thought of turning the alert into running a script but I don't know how to or if even if it is possible to get the log filename over to the script. If I could do that I could then send them the entire log file.
Anyone able to tell me if it is possible to get that data to the script?
I need help/ideas
I need help/ideas
You do not have the required permissions to view the files attached to this post.
2 of XI5.6.14 Prod/DR/DEV - Nagios LogServer 2 Nodes
See my projects on the Exchange at BanditBBS - Also check out my Nagios stuff on my personal page at Bandit's Home and at github
See my projects on the Exchange at BanditBBS - Also check out my Nagios stuff on my personal page at Bandit's Home and at github
Re: I need help/ideas
Unfortunately only the following marcos can be passed to scripts:
One option worth checking out is creating a script which queries elasticsearch directly when it is run and through this you could parse out the filename. Please share a copy of the query if you need help looking into this further.%count% - The total # of events
%status% - The status (ok, warning, critical)
%output% - The output from the alert
%lastrun% - The timestamp of the last run
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
Re: I need help/ideas
Here is the query. I can figure out the script for the most part, just unsure how to query elasticsearch directly. shell or perl is fine for me if you can help with that part!
Code: Select all
{
"query": {
"filtered": {
"query": {
"bool": {
"should": [
{
"query_string": {
"query": "\"Invalid collector_id\" \"Invalid lender_vendor_id: (passed value)\" \"Invalid company code for (passed value)\" \"Either $lender_vendor_id OR $repo_vendor is required\" \"Invalid phone type\" \"User $asid does not have permission to add updates\""
}
},
{
"query_string": {
"query": "\"missing invoice id for Invoice Info\" \"Few duplicate invoices are Rejected\""
}
},
{
"query_string": {
"query": "\"Invalid Reason Passed [n]. A valid reason type string, OR reason type code must be passed\" \"Invalid Reason Type Passed [n]. A valid reason type string, OR reason type code must be passed\" \"Case can not be closed it is currently in {$status} status.\""
}
},
{
"query_string": {
"query": "\"Your event request ID exceeds the maximum number. Please try a more recent event id.\""
}
},
{
"query_string": {
"query": "\"Vendor not found [V05080]\""
}
},
{
"query_string": {
"query": "\"Serious ERROR [Unhandled Exception] Invalid enumeration: phone_type passed as \\\"6\\\", should be one of: \\\"1\\\",\\\"2\\\"\""
}
},
{
"query_string": {
"query": "\"Collateral year, make, model AND vin are required\""
}
},
{
"query_string": {
"query": "\"Serious ERROR [Unhandled Exception] Invalid enumeration: bankruptcy_type passed as \\\"e\\\", should be one of: \\\"0\\\",\\\"1\\\",\\\"2\\\",\\\"3\\\",\\\"4\\\",\\\"5\\\",\\\"6\\\",\\\"7\\\",\\\"8\\\",\\\"9\\\",\\\"10\\\",\\\"11\\\",\\\"12\\\",\\\"13\\\",\\\"14\\\",\\\"15\\\",\\\"16\\\",\\\"17\\\",\\\"18\\\",\\\"19\\\",\\\"20\\\",\\\"21\\\",\\\"22\\\"\""
}
}
]
}
},
"filter": {
"bool": {
"must": [
{
"range": {
"@timestamp": {
"from": 1553789776417,
"to": 1553876176417
}
}
},
{
"fquery": {
"query": {
"query_string": {
"query": "logsource:(\"hdfs-chi-pdb01\")"
}
},
"_cache": true
}
},
{
"fquery": {
"query": {
"query_string": {
"query": "type:(\"RDN\")"
}
},
"_cache": true
}
}
],
"must_not": [
{
"fquery": {
"query": {
"query_string": {
"query": "filename:(\"/ofsh_prod/hdfs/ofsllprd/logs/alert.log\")"
}
},
"_cache": true
}
}
]
}
}
}
}
}2 of XI5.6.14 Prod/DR/DEV - Nagios LogServer 2 Nodes
See my projects on the Exchange at BanditBBS - Also check out my Nagios stuff on my personal page at Bandit's Home and at github
See my projects on the Exchange at BanditBBS - Also check out my Nagios stuff on my personal page at Bandit's Home and at github
Re: I need help/ideas
Try the following:
Note the change to the time range in the query - you'll want to adjust this accordingly per https://www.elastic.co/guide/en/elastic ... query.html
Code: Select all
curl -XPOST 'localhost:9200/logstash-*/_search?pretty' -d '{
"query": {
"filtered": {
"query": {
"bool": {
"should": [
{
"query_string": {
"query": "\"Invalid collector_id\" \"Invalid lender_vendor_id: (passed value)\" \"Invalid company code for (passed value)\" \"Either $lender_vendor_id OR $repo_vendor is required\" \"Invalid phone type\" \"User $asid does not have permission to add updates\""
}
},
{
"query_string": {
"query": "\"missing invoice id for Invoice Info\" \"Few duplicate invoices are Rejected\""
}
},
{
"query_string": {
"query": "\"Invalid Reason Passed [n]. A valid reason type string, OR reason type code must be passed\" \"Invalid Reason Type Passed [n]. A valid reason type string, OR reason type code must be passed\" \"Case can not be closed it is currently in {$status} status.\""
}
},
{
"query_string": {
"query": "\"Your event request ID exceeds the maximum number. Please try a more recent event id.\""
}
},
{
"query_string": {
"query": "\"Vendor not found [V05080]\""
}
},
{
"query_string": {
"query": "\"Serious ERROR [Unhandled Exception] Invalid enumeration: phone_type passed as \\\"6\\\", should be one of: \\\"1\\\",\\\"2\\\"\""
}
},
{
"query_string": {
"query": "\"Collateral year, make, model AND vin are required\""
}
},
{
"query_string": {
"query": "\"Serious ERROR [Unhandled Exception] Invalid enumeration: bankruptcy_type passed as \\\"e\\\", should be one of: \\\"0\\\",\\\"1\\\",\\\"2\\\",\\\"3\\\",\\\"4\\\",\\\"5\\\",\\\"6\\\",\\\"7\\\",\\\"8\\\",\\\"9\\\",\\\"10\\\",\\\"11\\\",\\\"12\\\",\\\"13\\\",\\\"14\\\",\\\"15\\\",\\\"16\\\",\\\"17\\\",\\\"18\\\",\\\"19\\\",\\\"20\\\",\\\"21\\\",\\\"22\\\"\""
}
}
]
}
},
"filter": {
"bool": {
"must": [
{
"range": {
"@timestamp": {
"from": "now-5h",
"to": "now"
}
}
},
{
"fquery": {
"query": {
"query_string": {
"query": "logsource:(\"hdfs-chi-pdb01\")"
}
},
"_cache": true
}
},
{
"fquery": {
"query": {
"query_string": {
"query": "type:(\"RDN\")"
}
},
"_cache": true
}
}
],
"must_not": [
{
"fquery": {
"query": {
"query_string": {
"query": "filename:(\"/ofsh_prod/hdfs/ofsllprd/logs/alert.log\")"
}
},
"_cache": true
}
}
]
}
}
}
}
}'As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
Re: I need help/ideas
Thanks! Tested and will work, now I just have to spend hours making the code I need, but this will definitely get me started!
You can close this and mark me happy
LOL
You can close this and mark me happy
LOL2 of XI5.6.14 Prod/DR/DEV - Nagios LogServer 2 Nodes
See my projects on the Exchange at BanditBBS - Also check out my Nagios stuff on my personal page at Bandit's Home and at github
See my projects on the Exchange at BanditBBS - Also check out my Nagios stuff on my personal page at Bandit's Home and at github
-
scottwilkerson
- DevOps Engineer
- Posts: 19396
- Joined: Tue Nov 15, 2011 3:11 pm
- Location: Nagios Enterprises
- Contact:
Re: I need help/ideas
great!BanditBBS wrote:Thanks! Tested and will work, now I just have to spend hours making the code I need, but this will definitely get me started!
You can close this and mark me happy