[dev.icinga.com #3159] core: log a message when reaper max time is reached

[icinga-migration]

This issue has been migrated from Redmine: https://dev.icinga.com/issues/3159

Created by mfriedrich on 2012-09-22 15:27:18 +00:00

Assignee: mfriedrich
Status: Resolved (closed on 2012-09-25 13:10:10 +00:00)
Target Version: 1.8
Last Update: 2012-09-25 13:10:10 +00:00 (in Redmine)

Icinga Version: 1.7.2
OS Version: Centos 6

---

discussed that with tom gelf, and we came to the conclusion, that reaching the max time for the reaper itsself and the core bailing out of reaping is rather servere in performance regards. this is due to the fact that the core cannot process the already stashed checkresults queue properly, and having problems to stash even more checkresults being read from disk over there. the root cause for that could be more than just a slow core - but it will of course increase latency, or even 100% cpu load and unresponsible cores.
so in order to stay sane, at least log a message (and not only debug message), to inform the user and point to some performance tuning tips too.

Changesets

2012-09-23 09:34:00 +00:00 by mfriedrich 50c70c6

core: log a message when reaper max time is reached #3159

this is still logged to icinga.debug, but in favor of letting the user
know in the first place, we'll spit a warning to syslog as well.

reaching the max reaper time means - core is not capable of processing
the checkresult list in memory, it gets longer and longer, and ever
since stashing checkresults onto this list does not happen by just
adding it (O(1)) at the end, but sorting the list by date, it will cause
worst case (O(n), decreasing overall performance just by the reaper not
being able to process more checkresults.
while this won't solve the issue, it will at least warn the user
directly that the reaper runs into performance issues. the current
reaped checkresult counter will be logged as well, in order to get an
idea what's going on. further performance analysis may then be required
by the user.

refs #3159

---

Relations:

• relates #3159

[icinga-migration]

Updated by mfriedrich on 2012-09-23 09:40:18 +00:00

• Status changed from Assigned to 7
• Done % changed from 0 to 100

[icinga-migration]

Updated by mfriedrich on 2012-09-25 13:10:10 +00:00

• Status changed from 7 to Resolved