[dev.icinga.com #12529] Icinga 2 sends SOFT recovery notifications

[icinga-migration]

This issue has been migrated from Redmine: https://dev.icinga.com/issues/12529

Created by phil_fry on 2016-08-24 07:02:44 +00:00

Assignee: gbeutner
Status: Resolved (closed on 2016-08-24 09:15:12 +00:00)
Target Version: 2.5.2
Last Update: 2016-08-30 07:18:32 +00:00 (in Redmine)

Icinga Version: 2.5.1
Backport?: Not yet backported
Include in Changelog: 1

---

Hi,

sorry that I have to reopen it (#12518), but the problem got worse. Now I'm getting LOTS of recovery notifications for soft {critical, warn, unknown} alerts.

Kind regards

Philippe

Changesets

2016-08-24 09:13:19 +00:00 by gbeutner 1a47a7c

Fix incorrect notifications for soft recoveries

fixes #12529

2016-08-24 09:50:29 +00:00 by gbeutner 220abc7

Fix incorrect notifications for soft recoveries

fixes #12529

2016-08-24 10:13:18 +00:00 by gbeutner 614e0ac

Fix incorrect notifications for soft recoveries

fixes #12529

2016-08-24 10:22:08 +00:00 by gbeutner deb938d

Fix incorrect notifications for soft recoveries

fixes #12529

2016-08-24 12:01:10 +00:00 by gbeutner 8c1fc81

Fix incorrect notifications for soft recoveries

fixes #12529

---

Relations:

• relates #12530
• relates #12529
• relates #12529

[icinga-migration]

Updated by phenomic on 2016-08-24 07:22:09 +00:00

Jep, same here. Getting notifications after soft state.
But i'm not getting recovery notifications - i'm getting problem notifications.

PROBLEM - vmware-esx9 - CPU_USAGE is OK

[2016-08-24 09:14:09 +0200] information/Checkable: Checking for configured notifications for object 'vmware-esx9!CPU_USAGE'
[2016-08-24 09:14:09 +0200] information/Notification: Sending 'Problem' notification 'vmware-esx9!CPU_USAGE!mail-icingaadmin for user 'xxxxxxx'
[2016-08-24 09:14:09 +0200] information/Notification: Completed sending 'Problem' notification 'vmware-esx9!CPU_USAGE!mail-icingaadmin' for checkable 'vmware-esx9!CPU_USAGE' and user 'xxxxxx'.

[icinga-migration]

Updated by Clasko on 2016-08-24 07:45:24 +00:00

Same here and also problem notifications but with state "OK".

[icinga-migration]

Updated by ios77 on 2016-08-24 08:12:31 +00:00

Same problem for me,
receiving a lot of recovery notifications for SOFT NOT-OK states.
Please fix it thanks.

[icinga-migration]

Updated by gbeutner on 2016-08-24 08:13:21 +00:00

• Status changed from New to Assigned
• Assigned to set to gbeutner
• Target Version set to 2.5.2

[icinga-migration]

Updated by gbeutner on 2016-08-24 09:13:38 +00:00

• Relates set to 12530

[icinga-migration]

Updated by gbeutner on 2016-08-24 09:15:12 +00:00

• Status changed from Assigned to Resolved
• Done % changed from 0 to 100

Applied in changeset 1a47a7c.

[icinga-migration]

Updated by mfriedrich on 2016-08-24 09:49:21 +00:00

• Priority changed from Normal to Urgent

[icinga-migration]

Updated by mfriedrich on 2016-08-24 10:02:19 +00:00

• Subject changed from getting spurious recovery notifications (2) to Icinga 2 sends SOFT recovery notifications

[icinga-migration]

Updated by mnardin on 2016-08-24 12:17:42 +00:00

Hi,
just to clarify: is this problem https://dev.icinga.org/issues/12293#change-53944 still fixed? I don't know anything about the source code, but I've noticed that part of the code regarding the previous fix was changed.

Best regards
Mirko

[icinga-migration]

Updated by mfriedrich on 2016-08-24 12:22:33 +00:00

• Relates set to 12293

[icinga-migration]

Updated by mfriedrich on 2016-08-24 12:23:53 +00:00

Nope that has been reverted. Broken fix slipped through the notification QA tests.

[icinga-migration]

Updated by mnardin on 2016-08-25 12:19:19 +00:00

Thank you, understood. What is the planned target version to fix this behavior (issue 12293) again?

[icinga-migration]

Updated by ios77 on 2016-08-26 10:01:41 +00:00

Dear, have you really fix this bug?
With the last update I have both notification for soft error anf for soft recovery!!!

[icinga-migration]

Updated by gbeutner on 2016-08-26 10:02:19 +00:00

ios77: icinga2 --version

[icinga-migration]

Updated by ios77 on 2016-08-26 10:04:22 +00:00

icinga2 - The Icinga 2 network monitoring daemon (version: v2.5.3)

Copyright © 2012-2016 Icinga Development Team (https://www.icinga.org/)
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl2.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Application information:
Installation root: /usr
Sysconf directory: /etc
Run directory: /run
Local state directory: /var
Package data directory: /usr/share/icinga2
State path: /var/lib/icinga2/icinga2.state
Modified attributes path: /var/lib/icinga2/modified-attributes.conf
Objects path: /var/cache/icinga2/icinga2.debug
Vars path: /var/cache/icinga2/icinga2.vars
PID path: /run/icinga2/icinga2.pid

System information:
Platform: Red Hat Enterprise Linux Workstation
Platform version: 7.2 (Maipo)
Kernel: Linux
Kernel version: 3.10.0-327.28.2.el7.x86_64
Architecture: x86_64

Build information:
Compiler: GNU 4.8.5
Build host: unknown

[icinga-migration]

Updated by gbeutner on 2016-08-26 10:05:24 +00:00

No, this should definitely work with 2.5.3.

[icinga-migration]

Updated by ios77 on 2016-08-26 10:13:19 +00:00

Service using this template start to send notification for soft problem state and for soft recovery in the last two icinga version (2.5.1-3)

template Service "fbk-critical-service" {

/* Maximum number of checks which are executed before changing to a hard state. */
max_check_attempts = 2
check_interval = 1m
retry_interval = 30s
}

no configuration changes or others. Only version update...
Maybe I am wrong, but in my opinion is not fixed.
Could you please double check it?

[icinga-migration]

Updated by gbeutner on 2016-08-26 10:37:12 +00:00

Are you able to reproduce this with max_check_attempts other than 2?

[icinga-migration]

Updated by ios77 on 2016-08-29 09:03:13 +00:00

Dear,
yes same problem with max_check_attempts = 3.
The notifications are sended before reaching the hard state.

[2016-08-29 10:49:54 +0200] information/Notification: Completed sending 'Problem' notification 'XXX.itmail-itid2' for checkable 'XXX.it!httpInaz' and user 'XXX'.
[2016-08-29 10:50:54 +0200] information/Notification: Completed sending 'Recovery' notification 'XXX.itmail-itid2' for checkable 'XXX.it!httpInaz' and user 'XXX'.

Thanks

[icinga-migration]

Updated by gbeutner on 2016-08-30 07:18:32 +00:00

There does appear to be a problem with max_check_attempts = 2 (#12530), however I am unable to reproduce this with any other value.

[icinga-migration]

Updated by mfriedrich on 2016-09-27 14:11:59 +00:00

• Relates set to 12694