New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[dev.icinga.com #10002] Deadlock in WorkQueue::Enqueue #3324
Comments
Updated by aledermueller on 2015-08-27 07:10:51 +00:00
The same thing happened again. Now the second master shows the same behavior/logs. A stacktrace of both is attached, master1 is the host writing to the ido-master. Thanks, Achim |
Updated by mfriedrich on 2015-08-27 14:52:46 +00:00
|
Updated by mfrosch on 2015-08-31 11:23:54 +00:00 Maybe also connected to #9976 ? |
Updated by mfrosch on 2015-08-31 11:24:00 +00:00
|
Updated by mfrosch on 2015-08-31 14:24:42 +00:00
|
Updated by gbeutner on 2015-09-02 05:46:59 +00:00 There's an experimental patch in the master branch which needs further testing. |
Updated by Anonymous on 2015-09-02 05:47:02 +00:00
Applied in changeset 5c77e6e. |
Updated by gbeutner on 2015-09-02 05:47:19 +00:00
|
Updated by mfriedrich on 2015-09-14 08:22:08 +00:00 According to Achim and Blerim, the fixes made it work again (2.3.10 without fixes causes trouble, the snapshot packages run fine for nearly a week now). I'd say we'll test this a little more and may back port that into 2.3.11 next week. |
Updated by mfriedrich on 2015-09-14 08:23:04 +00:00
|
Updated by mfriedrich on 2015-10-15 13:19:22 +00:00
|
This issue has been migrated from Redmine: https://dev.icinga.com/issues/10002
Created by aledermueller on 2015-08-26 13:34:46 +00:00
Assignee: gbeutner
Status: Resolved (closed on 2015-10-15 13:19:22 +00:00)
Target Version: 2.3.11
Last Update: 2015-10-15 13:19:22 +00:00 (in Redmine)
Hey,
Agents (zones): approx. 400 (mixed versions with 2.3.8 and 2.3.9)
Masters: 2 (Version 2.3.8)
After a while Icinga2 on one master hangs without using resources like CPU and IO. netstat shows full Recv-Qs (data from the agents) and empty Send-Qs. While 2/3 of the connections is on close_wait, the other 1/3 is established.
A stacktrace is attached: gdb -p xxx -ex 'thread apply all bt full' -ex deta -ex q -batch > debug
In the debug log are mainly the following entries. The counter for pending tasks is growing....
Thanks, Achim
Attachments
Changesets
2015-09-02 05:46:30 +00:00 by (unknown) 5c77e6e
2015-09-02 07:16:20 +00:00 by (unknown) 35acba7
2015-10-15 13:16:51 +00:00 by (unknown) e480af3
2015-10-15 13:18:02 +00:00 by (unknown) c8d24b6
Relations:
The text was updated successfully, but these errors were encountered: