Page 4 of 4
Re: Some alerts not firing
Posted: Thu Oct 08, 2015 4:57 pm
by jolson
Good stuff! I'll standby for your response.
Re: Some alerts not firing
Posted: Tue Oct 13, 2015 11:44 am
by Jklre
jolson wrote:Good stuff! I'll standby for your response.
Unfortunately it looks like this has not resolved the issue.
Audit logs
Code: Select all
2015-10-13T07:20:16.678-07:00 ALERT Alert Name 106929 CANADA: StdAssignmentDeliveryWF Error - - Priority 3/2 returned OK: 0 matching e...
View: Table / JSON / Raw
Field Action Value Search
_id AVBhkrenVjtz_tyMH_-k
_index nagioslogserver_log
_type ALERT
created 1444746016678
message Alert Name 106929 CANADA: StdAssignmentDeliveryWF Error - - Priority 3/2 returned OK: 0 matching entries found |logs=0;0;0
source Nagios Log Server
type ALERT
2015-10-13T07:04:26.630-07:00 ALERT Alert Name 106929 CANADA: StdAssignmentDeliveryWF Error - - Priority 3/2 returned OK: 0 matching e...
View: Table / JSON / Raw
Field Action Value Search
_id AVBhhDiHVjtz_tyMH_Qd
_index nagioslogserver_log
_type ALERT
created 1444745066630
message Alert Name 106929 CANADA: StdAssignmentDeliveryWF Error - - Priority 3/2 returned OK: 0 matching entries found |logs=0;0;0
source Nagios Log Server
type ALERT
2015-10-13T06:48:41.753-07:00 ALERT Alert Name 106929 CANADA: StdAssignmentDeliveryWF Error - - Priority 3/2 returned OK: 0 matching e...
View: Table / JSON / Raw
Field Action Value Search
_id AVBhdc2aVjtz_tyMH-nx
_index nagioslogserver_log
_type ALERT
created 1444744121753
message Alert Name 106929 CANADA: StdAssignmentDeliveryWF Error - - Priority 3/2 returned OK: 0 matching entries found |logs=0;0;0
source Nagios Log Server
type ALERT
Dashboard
error.jpg
Alert
Code: Select all
2015-10-13T07:04:59.396-07:00 172.26.20.140 syslog <11>pjbc16n1lxv jboss_jbc16: node1: [ID 23854 Oct 13, 2015 7:04:59 AM com.mitchell.services.core.errorlog.server.SyslogWriter write#012SEVERE:
@timestamp 2015-10-13T14:04:59.396Z
Re: Some alerts not firing
Posted: Tue Oct 13, 2015 11:54 am
by jolson
Do you have product support? I'd like to perform a remote session with you and get to the bottom of this, but you'd need to be a paying customer. I ask because this post is in the general forum.
Re: Some alerts not firing
Posted: Tue Oct 13, 2015 12:02 pm
by Jklre
Jklre wrote:jolson wrote:Good stuff! I'll standby for your response.
Unfortunately it looks like this has not resolved the issue.
Audit logs
Code: Select all
2015-10-13T07:20:16.678-07:00 ALERT Alert Name 106929 CANADA: StdAssignmentDeliveryWF Error - - Priority 3/2 returned OK: 0 matching e...
View: Table / JSON / Raw
Field Action Value Search
_id AVBhkrenVjtz_tyMH_-k
_index nagioslogserver_log
_type ALERT
created 1444746016678
message Alert Name 106929 CANADA: StdAssignmentDeliveryWF Error - - Priority 3/2 returned OK: 0 matching entries found |logs=0;0;0
source Nagios Log Server
type ALERT
2015-10-13T07:04:26.630-07:00 ALERT Alert Name 106929 CANADA: StdAssignmentDeliveryWF Error - - Priority 3/2 returned OK: 0 matching e...
View: Table / JSON / Raw
Field Action Value Search
_id AVBhhDiHVjtz_tyMH_Qd
_index nagioslogserver_log
_type ALERT
created 1444745066630
message Alert Name 106929 CANADA: StdAssignmentDeliveryWF Error - - Priority 3/2 returned OK: 0 matching entries found |logs=0;0;0
source Nagios Log Server
type ALERT
2015-10-13T06:48:41.753-07:00 ALERT Alert Name 106929 CANADA: StdAssignmentDeliveryWF Error - - Priority 3/2 returned OK: 0 matching e...
View: Table / JSON / Raw
Field Action Value Search
_id AVBhdc2aVjtz_tyMH-nx
_index nagioslogserver_log
_type ALERT
created 1444744121753
message Alert Name 106929 CANADA: StdAssignmentDeliveryWF Error - - Priority 3/2 returned OK: 0 matching entries found |logs=0;0;0
source Nagios Log Server
type ALERT
Dashboard
error.jpg
Alert
Code: Select all
2015-10-13T07:04:59.396-07:00 172.26.20.140 syslog <11>pjbc16n1lxv jboss_jbc16: node1: [ID 23854 Oct 13, 2015 7:04:59 AM com.mitchell.services.core.errorlog.server.SyslogWriter write#012SEVERE:
@timestamp 2015-10-13T14:04:59.396Z
Looking deeper into this. This is occurring on a different cluster than our main one. It has less alerts configured (948 instead of 2000)
Running the command
[root@pcanls01lxv ~]# ps -ef | egrep "jobs|poller" | grep -v grep | wc -l
12
[root@pcanls01lxv ~]# ps -ef | egrep "jobs|poller" | grep -v grep | wc -l
10
[root@pcanls01lxv ~]# ps -ef | egrep "jobs|poller" | grep -v grep | wc -l
12
[root@pcanls01lxv ~]# ps -ef | egrep "jobs|poller" | grep -v grep | wc -l
15
as you can see the jobs are at 12 + not sure whats causing this. I'm going to bounce the cluster and see if it gets any better.
Re: Some alerts not firing
Posted: Tue Oct 13, 2015 12:09 pm
by Jklre
After the bounce:
[root@pcanls01lxv ~]# ps -ef | egrep "jobs|poller" | grep -v grep | wc -l
2
[root@pcanls01lxv ~]# ps -ef | egrep "jobs|poller" | grep -v grep | wc -l
6
[root@pcanls01lxv ~]# ps -ef | egrep "jobs|poller" | grep -v grep | wc -l
4
Re: Some alerts not firing
Posted: Tue Oct 13, 2015 1:10 pm
by Jklre
jolson wrote:Do you have product support? I'd like to perform a remote session with you and get to the bottom of this, but you'd need to be a paying customer. I ask because this post is in the general forum.
yes we do, but for some reason my account does not have access to that forum.
Re: Some alerts not firing
Posted: Tue Oct 13, 2015 4:41 pm
by jolson
yes we do, but for some reason my account does not have access to that forum.
I'll look into that - would you please send an email to
[email protected] and reference this thread? I'll pick up your ticket and we can take an in-depth look at this with a developer if necessary.
Jesse
Re: Some alerts not firing
Posted: Wed Oct 14, 2015 11:45 am
by Jklre
jolson wrote:yes we do, but for some reason my account does not have access to that forum.
I'll look into that - would you please send an email to
[email protected] and reference this thread? I'll pick up your ticket and we can take an in-depth look at this with a developer if necessary.
Jesse
I went ahead and opened it up a ticket with you guys.
Thank you
Re: Some alerts not firing
Posted: Wed Oct 14, 2015 11:55 am
by jolson
I'll go ahead and lock this thread - when we come to a resolution I'll be sure to add that here.