[dev.icinga.com #10226] Icinga2 reload timeout results in killing old and new process because of systemd

[icinga-migration]

This issue has been migrated from Redmine: https://dev.icinga.com/issues/10226

Created by dgoetz on 2015-09-28 08:15:55 +00:00

Assignee: gbeutner
Status: Resolved (closed on 2015-12-16 15:05:03 +00:00)
Target Version: 2.4.2
Last Update: 2016-02-23 09:58:15 +00:00 (in Redmine)

Icinga Version: 2.3.10
Backport?: Already backported
Include in Changelog: 1

---

The reload mechanismen send the old process a SIGKILL signal if it fails to stop before timeout.
The old process normally changes the pidfile to contain the new processes pid, this does not happen if killed.
So systemd kills all processes in the cgroup matching the old pid including the new process which results in no active icinga 2 process.

So effectively a reload can result in a stopped monitoring

Attachments

• i2_10226_timeline.JPG mfriedrich - 2015-12-16 15:47:22 +00:00

Changesets

2015-12-16 14:57:59 +00:00 by mfriedrich 8b6d335

Fix failed reload via systemd

fixes #10226

2016-02-23 08:16:01 +00:00 by mfriedrich 99efc7b

Fix failed reload via systemd

fixes #10226

---

Relations:

• relates #9312
• relates #10226

[icinga-migration]

Updated by mfriedrich on 2015-09-28 08:20:50 +00:00

• Relates set to 9312

[icinga-migration]

Updated by mfriedrich on 2015-10-26 16:39:27 +00:00

• Category set to libbase

Please try the suggested workaround in https://dev.icinga.org/issues/9312#note-8

[icinga-migration]

Updated by dgoetz on 2015-10-27 07:18:37 +00:00

Already tried setting the systemd timeouts for start and stop, but it still times out after 60 seconds which is the timeout of icinga we discussed.

[icinga-migration]

Updated by mfriedrich on 2015-10-28 08:46:13 +00:00

Notes:

• TerminateAndWaitForEnd()
• Application::GetInstance()->Run();
• UpdatePidFile

[icinga-migration]

Updated by mfriedrich on 2015-10-28 08:46:28 +00:00

• Target Version set to Backlog

[icinga-migration]

Updated by mfriedrich on 2015-10-29 08:55:19 +00:00

• Relates set to 7752

[icinga-migration]

Updated by mfriedrich on 2015-11-03 09:41:22 +00:00

• Estimated Hours set to 32

[icinga-migration]

Updated by mfriedrich on 2015-11-25 16:17:53 +00:00

• Priority changed from Normal to High

[icinga-migration]

Updated by mfriedrich on 2015-12-15 16:57:30 +00:00

• Target Version changed from Backlog to 2.4.2

[icinga-migration]

Updated by mfriedrich on 2015-12-16 15:00:00 +00:00

• Status changed from New to Assigned
• Assigned to set to gbeutner

[icinga-migration]

Updated by mfriedrich on 2015-12-16 15:05:03 +00:00

• Status changed from Assigned to Resolved
• Done % changed from 0 to 100

Applied in changeset 8b6d335.

[icinga-migration]

Updated by mfriedrich on 2015-12-16 15:47:32 +00:00

• File added i2_10226_timeline.JPG

[icinga-migration]

Updated by gbeutner on 2016-02-23 09:58:15 +00:00

• Backport? changed from Not yet backported to Already backported