[dev.icinga.com #12127] DB IDO query queue does not clean up with v2.4.10-520-g124c80b #4356

icinga-migration · 2016-07-07T16:20:35Z

This issue has been migrated from Redmine: https://dev.icinga.com/issues/12127

Created by mfriedrich on 2016-07-07 16:20:35 +00:00

Assignee: gbeutner
Status: Resolved (closed on 2016-07-12 10:00:06 +00:00)
Target Version: 2.5.0
Last Update: 2016-07-12 10:03:56 +00:00 (in Redmine)

Icinga Version: v2.4.10-520-g124c80b
Backport?: Not yet backported
Include in Changelog: 0

mbmif /usr/local/icinga2/etc/icinga2/conf.d (master) # icinga2 daemon
[2016-07-07 18:02:51 +0200] information/cli: Icinga application loader (version: v2.4.10-520-g124c80b; debug)
[2016-07-07 18:02:51 +0200] information/cli: Loading configuration file(s).
[2016-07-07 18:02:51 +0200] information/ConfigItem: Committing config item(s).
[2016-07-07 18:02:51 +0200] information/ApiListener: My API identity: mbmif.int.netways.de
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'satellite-host' (in /usr/local/icinga2/etc/icinga2/conf.d/satellite.conf: 29:1-29:41) for type 'Dependency' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-freef:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-freeg:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-freeh:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-freei:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-freej:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-freek:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-freel:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-freem:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-freen:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-freeo:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-freep:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-freeq:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-freer:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-frees:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-freet:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-freeu:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-freev:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-freew:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-freex:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-freey:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] warning/ApplyRule: Apply rule 'wmi-disk-freez:' (in /usr/local/icinga2/etc/icinga2/tests/for-apply.conf: 4:9-4:60) for type 'Service' does not match anywhere!
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 2 Dependencies.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 9 TimePeriods.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 19 Users.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 68 Hosts.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 10 HostGroups.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 1 IcingaApplication.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 175 CheckCommands.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 5 NotificationCommands.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 29 Notifications.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 1 CheckerComponent.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 1 LivestatusListener.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 1 NotificationComponent.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 1 FileLogger.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 1 Endpoint.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 3 Zones.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 1 ApiListener.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 3 ApiUsers.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 3 Comments.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 170 Services.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 1 ScheduledDowntime.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 4 UserGroups.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 5 ServiceGroups.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 1 Downtime.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 1 ExternalCommandListener.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 1 StatusDataWriter.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Instantiated 1 IdoMysqlConnection.
[2016-07-07 18:02:52 +0200] information/ScriptGlobal: Dumping variables to file '/usr/local/icinga2/var/cache/icinga2/icinga2.vars'
[2016-07-07 18:02:52 +0200] information/ConfigObject: Restoring program state from file '/usr/local/icinga2/var/lib/icinga2/icinga2.state'
[2016-07-07 18:02:52 +0200] information/ConfigObject: Restored 517 objects. Loaded 0 new objects without state.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Triggering Start signal for config items
[2016-07-07 18:02:52 +0200] information/ApiListener: Copying 3 zone configuration files for zone 'global-templates' to '/usr/local/icinga2/var/lib/icinga2/api/zones/global-templates'.
[2016-07-07 18:02:52 +0200] information/ApiListener: Adding new listener on port '5665'
[2016-07-07 18:02:52 +0200] information/LivestatusListener: Created UNIX socket in '/usr/local/icinga2/var/run/icinga2/cmd/livestatus'.
[2016-07-07 18:02:52 +0200] information/ConfigItem: Activated all objects.
[2016-07-07 18:02:52 +0200] information/DbConnection: Resuming IDO connection: ido-mysql
[2016-07-07 18:02:52 +0200] information/IdoMysqlConnection: MySQL IDO instance id: 1 (schema version: '1.14.1')
[2016-07-07 18:02:52 +0200] information/IdoMysqlConnection: Finished reconnecting to MySQL IDO database in 0.409111 second(s).
[2016-07-07 18:02:57 +0200] information/Notification: Sending notification 'mbmif.int.netways.de!ido-pgsql!mail-icingaadmin' for user 'icingaadmin'
[2016-07-07 18:02:57 +0200] information/Notification: Completed sending notification 'mbmif.int.netways.de!ido-pgsql!mail-icingaadmin' for checkable 'mbmif.int.netways.de!ido-pgsql' and user 'icingaadmin'.
[2016-07-07 18:02:57 +0200] information/Notification: Sending notification 'mbmif.int.netways.de!disk!mail-icingaadmin' for user 'icingaadmin'
[2016-07-07 18:02:57 +0200] information/Notification: Sending notification 'mbmif.int.netways.de!disk /!mail-icingaadmin' for user 'icingaadmin'
[2016-07-07 18:02:57 +0200] information/Notification: Completed sending notification 'mbmif.int.netways.de!disk!mail-icingaadmin' for checkable 'mbmif.int.netways.de!disk' and user 'icingaadmin'.
[2016-07-07 18:02:57 +0200] information/Notification: Sending notification 'mbmif.int.netways.de3!disk!mail-icingaadmin' for user 'icingaadmin'
[2016-07-07 18:02:57 +0200] information/Notification: Completed sending notification 'mbmif.int.netways.de!disk /!mail-icingaadmin' for checkable 'mbmif.int.netways.de!disk /' and user 'icingaadmin'.
[2016-07-07 18:02:57 +0200] information/Notification: Sending notification 'mbmif.int.netways.de3!disk /!mail-icingaadmin' for user 'icingaadmin'
[2016-07-07 18:02:57 +0200] information/Notification: Completed sending notification 'mbmif.int.netways.de3!disk!mail-icingaadmin' for checkable 'mbmif.int.netways.de3!disk' and user 'icingaadmin'.
[2016-07-07 18:02:57 +0200] information/Notification: Sending notification 'mbmif.int.netways.de!procs!mail-icingaadmin' for user 'icingaadmin'
[2016-07-07 18:02:57 +0200] information/Notification: Completed sending notification 'mbmif.int.netways.de3!disk /!mail-icingaadmin' for checkable 'mbmif.int.netways.de3!disk /' and user 'icingaadmin'.
[2016-07-07 18:02:57 +0200] information/Notification: Sending notification 'mbmif.int.netways.de!swap!mail-icingaadmin' for user 'icingaadmin'
[2016-07-07 18:02:57 +0200] information/Notification: Sending notification 'mbmif.int.netways.de!Linux Disk "/"!mail-icingaadmin' for user 'icingaadmin'
[2016-07-07 18:02:57 +0200] information/Notification: Completed sending notification 'mbmif.int.netways.de!procs!mail-icingaadmin' for checkable 'mbmif.int.netways.de!procs' and user 'icingaadmin'.
[2016-07-07 18:02:57 +0200] information/Notification: Completed sending notification 'mbmif.int.netways.de!Linux Disk "/"!mail-icingaadmin' for checkable 'mbmif.int.netways.de!Linux Disk "/"' and user 'icingaadmin'.
[2016-07-07 18:02:57 +0200] information/Notification: Completed sending notification 'mbmif.int.netways.de!swap!mail-icingaadmin' for checkable 'mbmif.int.netways.de!swap' and user 'icingaadmin'.
[2016-07-07 18:02:57 +0200] information/Checkable: Checking for configured notifications for object '11686-host'
[2016-07-07 18:02:57 +0200] information/Checkable: Checkable '11686-host' does not have any notifications.
[2016-07-07 18:03:03 +0200] information/Checkable: Checking for configured notifications for object 'flapping-host'
[2016-07-07 18:03:03 +0200] information/Checkable: Checkable 'flapping-host' does not have any notifications.
[2016-07-07 18:03:07 +0200] information/IdoMysqlConnection: Query queue items: 176, query rate: 14.2333/s (854/min 854/5min 854/15min); empty in infinite time, your database isn't able to keep up
[2016-07-07 18:03:08 +0200] information/Checkable: Checking for configured notifications for object '11686-host'
[2016-07-07 18:03:08 +0200] information/Checkable: Checkable '11686-host' does not have any notifications.
[2016-07-07 18:03:19 +0200] information/Checkable: Checking for configured notifications for object '11686-host'
[2016-07-07 18:03:19 +0200] information/Checkable: Checkable '11686-host' does not have any notifications.
[2016-07-07 18:03:21 +0200] information/Checkable: Checking for configured notifications for object 'mbmif.int.netways.de!load'
[2016-07-07 18:03:22 +0200] information/IdoMysqlConnection: Query queue items: 176, query rate: 16.0833/s (965/min 965/5min 965/15min); empty in infinite time, your database isn't able to keep up
[2016-07-07 18:03:24 +0200] information/Checkable: Checking for configured notifications for object 'flapping-host'
[2016-07-07 18:03:24 +0200] information/Checkable: Checkable 'flapping-host' does not have any notifications.
[2016-07-07 18:03:25 +0200] information/Checkable: Checking for configured notifications for object '11686-host'
[2016-07-07 18:03:25 +0200] information/Checkable: Checkable '11686-host' does not have any notifications.
[2016-07-07 18:03:35 +0200] information/Checkable: Checking for configured notifications for object '11686-host'
[2016-07-07 18:03:35 +0200] information/Checkable: Checkable '11686-host' does not have any notifications.
[2016-07-07 18:03:37 +0200] information/IdoMysqlConnection: Query queue items: 176, query rate: 18.3667/s (1102/min 1102/5min 1102/15min); empty in infinite time, your database isn't able to keep up
[2016-07-07 18:03:40 +0200] information/Checkable: Checking for configured notifications for object '11686-host'
[2016-07-07 18:03:40 +0200] information/Checkable: Checkable '11686-host' does not have any notifications.
[2016-07-07 18:03:45 +0200] information/Checkable: Checking for configured notifications for object '11686-host'
[2016-07-07 18:03:45 +0200] information/Checkable: Checkable '11686-host' does not have any notifications.
[2016-07-07 18:03:49 +0200] warning/PluginCheckTask: Check command for object 'mbmif.int.netways.de!swap' (PID: 26519, arguments: '/usr/local/sbin/check_swap' '-c' '25%' '-w' '50%') terminated with exit code 128, output: execvpe(/usr/local/sbin/check_swap) failed: No such file or directory

[2016-07-07 18:03:49 +0200] information/Checkable: Checking for configured notifications for object 'many-test-5!many-test-7'
[2016-07-07 18:03:49 +0200] information/Checkable: Checkable 'many-test-5!many-test-7' does not have any notifications.
[2016-07-07 18:03:49 +0200] information/Checkable: Checking for configured notifications for object 'many-test-7!many-test-9'
[2016-07-07 18:03:49 +0200] information/Checkable: Checkable 'many-test-7!many-test-9' does not have any notifications.
[2016-07-07 18:03:50 +0200] information/Checkable: Checking for configured notifications for object 'many-test-1'
[2016-07-07 18:03:50 +0200] information/Checkable: Checkable 'many-test-1' does not have any notifications.
[2016-07-07 18:03:50 +0200] information/Checkable: Checking for configured notifications for object 'many-test-7'
[2016-07-07 18:03:50 +0200] information/Checkable: Checkable 'many-test-7' does not have any notifications.
[2016-07-07 18:03:50 +0200] warning/PluginCheckTask: Check command for object 'test' (PID: 26553, arguments: 'sh' '-c' '/usr/local/icinga2/etc/icinga2/scripts/test.sh') terminated with exit code 127, output: sh: /usr/local/icinga2/etc/icinga2/scripts/test.sh: No such file or directory

[2016-07-07 18:03:50 +0200] information/Checkable: Checking for configured notifications for object 'many-test-4'
[2016-07-07 18:03:50 +0200] information/Checkable: Checkable 'many-test-4' does not have any notifications.
[2016-07-07 18:03:50 +0200] information/Checkable: Checking for configured notifications for object 'many-test-1!many-test-6'
[2016-07-07 18:03:50 +0200] information/Checkable: Checkable 'many-test-1!many-test-6' does not have any notifications.
[2016-07-07 18:03:51 +0200] information/Checkable: Checking for configured notifications for object 'many-test-6!many-test-3'
[2016-07-07 18:03:51 +0200] information/Checkable: Checkable 'many-test-6!many-test-3' does not have any notifications.
[2016-07-07 18:03:51 +0200] information/Checkable: Checking for configured notifications for object 'many-test-2!many-test-4'
[2016-07-07 18:03:51 +0200] information/Checkable: Checkable 'many-test-2!many-test-4' does not have any notifications.
[2016-07-07 18:03:52 +0200] information/IdoMysqlConnection: Query queue items: 176, query rate: 20.25/s (1215/min 1215/5min 1215/15min); empty in infinite time, your database isn't able to keep up
[2016-07-07 18:03:59 +0200] information/Checkable: Checking for configured notifications for object '11686-host'
[2016-07-07 18:03:59 +0200] information/Checkable: Checkable '11686-host' does not have any notifications.
[2016-07-07 18:04:04 +0200] information/Checkable: Checking for configured notifications for object '7355-host'
[2016-07-07 18:04:04 +0200] information/Checkable: Checkable '7355-host' does not have any notifications.
[2016-07-07 18:04:04 +0200] information/Checkable: Checking for configured notifications for object '11686-host'
[2016-07-07 18:04:04 +0200] information/Checkable: Checkable '11686-host' does not have any notifications.
[2016-07-07 18:04:07 +0200] information/IdoMysqlConnection: Query queue items: 176, query rate: 8.1/s (486/min 1340/5min 1340/15min); empty in infinite time, your database isn't able to keep up

MariaDB [icinga]> select count(*) from icinga_objects;
+----------+
| count(*) |
+----------+
|      469 |
+----------+
1 row in set (0.00 sec)

MariaDB [icinga]> Bye
michi@mbmif ~/coding/icinga/icinga2 (master) $ icinga2 --version
icinga2 - The Icinga 2 network monitoring daemon (version: v2.4.10-520-g124c80b; debug)

Copyright (c) 2012-2016 Icinga Development Team (https://www.icinga.org/)
License GPLv2+: GNU GPL version 2 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Application information:
  Installation root: /usr/local/icinga2
  Sysconf directory: /usr/local/icinga2/etc
  Run directory: /usr/local/icinga2/var/run
  Local state directory: /usr/local/icinga2/var
  Package data directory: /usr/local/icinga2/share/icinga2
  State path: /usr/local/icinga2/var/lib/icinga2/icinga2.state
  Modified attributes path: /usr/local/icinga2/var/lib/icinga2/modified-attributes.conf
  Objects path: /usr/local/icinga2/var/cache/icinga2/icinga2.debug
  Vars path: /usr/local/icinga2/var/cache/icinga2/icinga2.vars
  PID path: /usr/local/icinga2/var/run/icinga2/icinga2.pid

System information:
  Platform: Mac OS X
  Platform version: 10.11.5
  Kernel: Darwin
  Kernel version: 15.5.0
  Architecture: x86_64

Attachments

gdb_sleep_on_startup.txt tobiasvdk - 2016-07-11 16:10:34 +00:00

Changesets

2016-07-12 09:44:36 +00:00 by gbeutner ea57f00

Fix deadlock in DbObject::SendVars{Config,Status}Update

fixes #12127
fixes #12085

Relations:

relates #12127

The text was updated successfully, but these errors were encountered:

icinga-migration · 2016-07-11T16:11:57Z

Updated by tobiasvdk on 2016-07-11 16:11:57 +00:00

File added gdb_sleep_on_startup.txt

I sometimes see a weird behavior that the query rate stays at a very low level "forever" (until I restart icinga 2), e.g.

[2016-07-06 10:32:39 +0000] information/IdoMysqlConnection: Query queue items: 1000000, query rate: 0.683333/s (41/min 41/5min 41/15min); empty in infinite time, your database isn't able to keep up

$ sudo icinga2 --version
icinga2 - The Icinga 2 network monitoring daemon (version: r2.4.10-517-gf87f20e)

Copyright (c) 2012-2016 Icinga Development Team (https://www.icinga.org/)
License GPLv2+: GNU GPL version 2 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Application information:
  Installation root: /usr
  Sysconf directory: /etc
  Run directory: /run
  Local state directory: /var
  Package data directory: /usr/share/icinga2
  State path: /var/lib/icinga2/icinga2.state
  Modified attributes path: /var/lib/icinga2/modified-attributes.conf
  Objects path: /var/cache/icinga2/icinga2.debug
  Vars path: /var/cache/icinga2/icinga2.vars
  PID path: /run/icinga2/icinga2.pid

System information:
  Platform: Debian GNU/Linux
  Platform version: 8 (jessie)
  Kernel: Linux
  Kernel version: 4.6.0-0.bpo.1-amd64
  Architecture: x86_64

icinga-migration · 2016-07-12T09:46:11Z

Updated by gbeutner on 2016-07-12 09:46:11 +00:00

Relates set to 12085

icinga-migration · 2016-07-12T09:46:26Z

Updated by gbeutner on 2016-07-12 09:46:26 +00:00

Status changed from New to Assigned
Assigned to set to gbeutner

icinga-migration · 2016-07-12T09:48:07Z

Updated by mfriedrich on 2016-07-12 09:48:07 +00:00

[2016-07-12 11:46:42 +0200] information/IdoMysqlConnection: Query queue items: 175, query rate: 7.48333/s (449/min 1804/5min 1804/15min); empty in infinite time, your database isn't able to keep up
[2016-07-12 11:46:45 +0200] information/Checkable: Checking for configured notifications for object '11686-host'
[2016-07-12 11:46:45 +0200] information/Checkable: Checkable '11686-host' does not have any notifications.
[2016-07-12 11:46:57 +0200] information/IdoMysqlConnection: Query queue items: 175, query rate: 7.5/s (450/min 1918/5min 1918/15min); empty in infinite time, your database isn't able to keep up
[2016-07-12 11:46:59 +0200] information/Checkable: Checking for configured notifications for object '11686-host'
[2016-07-12 11:46:59 +0200] information/Checkable: Checkable '11686-host' does not have any notifications.
Process 11224 stopped
* thread #1: tid = 0x8dd61b, 0x00007fff8678210a libsystem_kernel.dylib`__semwait_signal + 10, stop reason = signal SIGSTOP
    frame #0: 0x00007fff8678210a libsystem_kernel.dylib`__semwait_signal + 10
libsystem_kernel.dylib`__semwait_signal:
->  0x7fff8678210a <+10>: jae    0x7fff86782114            ; <+20>
    0x7fff8678210c <+12>: movq   %rax, %rdi
    0x7fff8678210f <+15>: jmp    0x7fff8677c7f2            ; cerror
    0x7fff86782114 <+20>: retq
(lldb) bt all
* thread #1: tid = 0x8dd61b, 0x00007fff8678210a libsystem_kernel.dylib`__semwait_signal + 10, stop reason = signal SIGSTOP
  * frame #0: 0x00007fff8678210a libsystem_kernel.dylib`__semwait_signal + 10
    frame #1: 0x00007fff9a291d17 libsystem_c.dylib`nanosleep + 199
    frame #2: 0x00007fff9a291c0a libsystem_c.dylib`usleep + 54
    frame #3: 0x0000000101119ecb libbase.dylib`icinga::Utility::Sleep(timeout=2.5) + 43 at utility.cpp:424
    frame #4: 0x0000000100f0e873 libbase.dylib`icinga::Application::RunEventLoop(this=0x0000000101e66990) + 115 at application.cpp:282
    frame #5: 0x000000010429bc4a libicinga.dylib`icinga::IcingaApplication::Main(this=0x0000000101e66990) + 1386 at icingaapplication.cpp:119
    frame #6: 0x0000000100f13582 libbase.dylib`icinga::Application::Run(this=0x0000000101e66990) + 674 at application.cpp:894
    frame #7: 0x000000010032c81a libcli.dylib`icinga::DaemonCommand::Run(this=0x0000000101b02a50, vm=0x00007fff5fbfe710, ap=0x00007fff5fbfca68) const + 9050 at daemoncommand.cpp:314
    frame #8: 0x000000010000cfae icinga2`Main() + 28414 at icinga.cpp:484
    frame #9: 0x000000010000d3c4 icinga2`main(argc=2, argv=0x00007fff5fbff7a0) + 260 at icinga.cpp:750
    frame #10: 0x00007fff96c4f5ad libdyld.dylib`start + 1

  thread #2: tid = 0x8dd62e, 0x00007fff86781db6 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #0: 0x00007fff86781db6 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #1: 0x00007fff8e672728 libsystem_pthread.dylib`_pthread_cond_wait + 767
    frame #2: 0x00000001010c9208 libbase.dylib`boost::condition_variable::wait(this=0x00000001011a0cd8, m=0x0000700000080a68) + 152 at condition_variable.hpp:77
    frame #3: 0x00000001010f2cc1 libbase.dylib`icinga::ThreadPool::WorkerThread::ThreadProc(this=0x00000001011a0e00, queue=0x00000001011a0c98) + 1873 at threadpool.cpp:111
    frame #4: 0x00000001010fbaa1 libbase.dylib`boost::_mfi::mf1::operator(this=0x0000000101b08730, t=0x00000001011a0e00, a1=0x00000001011a0c98)(icinga::ThreadPool::WorkerThread&, icinga::ThreadPool::Queue&) const + 113 at mem_fn_template.hpp:186
    frame #5: 0x00000001010fb9fc libbase.dylib`void boost::_bi::list2, boost::reference_wrapper >::operator(this=0x0000000101b08740, (null)=type @ 0x0000700000080e48, f=0x0000000101b08730, a=0x0000700000080e70, (null)=0), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) + 108 at bind.hpp:319
    frame #6: 0x00000001010fb98a libbase.dylib`boost::_bi::bind_t, boost::_bi::list2, boost::reference_wrapper > >::operator(this=0x0000000101b08730)() + 58 at bind.hpp:1222
    frame #7: 0x00000001010fb91c libbase.dylib`boost::detail::thread_data, boost::_bi::list2, boost::reference_wrapper > > >::run(this=0x0000000101b08530) + 28 at thread.hpp:116
    frame #8: 0x000000010007f804 libboost_thread-mt.dylib`boost::(anonymous namespace)::thread_proxy(void*) + 164
    frame #9: 0x00007fff8e67199d libsystem_pthread.dylib`_pthread_body + 131
    frame #10: 0x00007fff8e67191a libsystem_pthread.dylib`_pthread_start + 168
    frame #11: 0x00007fff8e66f351 libsystem_pthread.dylib`thread_start + 13

  thread #3: tid = 0x8dd62f, 0x00007fff86781db6 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #0: 0x00007fff86781db6 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #1: 0x00007fff8e672728 libsystem_pthread.dylib`_pthread_cond_wait + 767
    frame #2: 0x00000001010c9208 libbase.dylib`boost::condition_variable::wait(this=0x00000001011a1040, m=0x0000700000103a68) + 152 at condition_variable.hpp:77
    frame #3: 0x00000001010f2cc1 libbase.dylib`icinga::ThreadPool::WorkerThread::ThreadProc(this=0x00000001011a1168, queue=0x00000001011a1000) + 1873 at threadpool.cpp:111
    frame #4: 0x00000001010fbaa1 libbase.dylib`boost::_mfi::mf1::operator(this=0x0000000101b089a0, t=0x00000001011a1168, a1=0x00000001011a1000)(icinga::ThreadPool::WorkerThread&, icinga::ThreadPool::Queue&) const + 113 at mem_fn_template.hpp:186
    frame #5: 0x00000001010fb9fc libbase.dylib`void boost::_bi::list2, boost::reference_wrapper >::operator(this=0x0000000101b089b0, (null)=type @ 0x0000700000103e48, f=0x0000000101b089a0, a=0x0000700000103e70, (null)=0), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) + 108 at bind.hpp:319
    frame #6: 0x00000001010fb98a libbase.dylib`boost::_bi::bind_t, boost::_bi::list2, boost::reference_wrapper > >::operator(this=0x0000000101b089a0)() + 58 at bind.hpp:1222
    frame #7: 0x00000001010fb91c libbase.dylib`boost::detail::thread_data, boost::_bi::list2, boost::reference_wrapper > > >::run(this=0x0000000101b087a0) + 28 at thread.hpp:116
    frame #8: 0x000000010007f804 libboost_thread-mt.dylib`boost::(anonymous namespace)::thread_proxy(void*) + 164
    frame #9: 0x00007fff8e67199d libsystem_pthread.dylib`_pthread_body + 131
    frame #10: 0x00007fff8e67191a libsystem_pthread.dylib`_pthread_start + 168
    frame #11: 0x00007fff8e66f351 libsystem_pthread.dylib`thread_start + 13

  thread #4: tid = 0x8dd630, 0x00007fff86781db6 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #0: 0x00007fff86781db6 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #1: 0x00007fff8e672728 libsystem_pthread.dylib`_pthread_cond_wait + 767
    frame #2: 0x00000001010c9208 libbase.dylib`boost::condition_variable::wait(this=0x00000001011a13a8, m=0x0000700000186a68) + 152 at condition_variable.hpp:77
    frame #3: 0x00000001010f2cc1 libbase.dylib`icinga::ThreadPool::WorkerThread::ThreadProc(this=0x00000001011a14d0, queue=0x00000001011a1368) + 1873 at threadpool.cpp:111
    frame #4: 0x00000001010fbaa1 libbase.dylib`boost::_mfi::mf1::operator(this=0x0000000101b08c10, t=0x00000001011a14d0, a1=0x00000001011a1368)(icinga::ThreadPool::WorkerThread&, icinga::ThreadPool::Queue&) const + 113 at mem_fn_template.hpp:186
    frame #5: 0x00000001010fb9fc libbase.dylib`void boost::_bi::list2, boost::reference_wrapper >::operator(this=0x0000000101b08c20, (null)=type @ 0x0000700000186e48, f=0x0000000101b08c10, a=0x0000700000186e70, (null)=0), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) + 108 at bind.hpp:319
    frame #6: 0x00000001010fb98a libbase.dylib`boost::_bi::bind_t, boost::_bi::list2, boost::reference_wrapper > >::operator(this=0x0000000101b08c10)() + 58 at bind.hpp:1222
    frame #7: 0x00000001010fb91c libbase.dylib`boost::detail::thread_data, boost::_bi::list2, boost::reference_wrapper > > >::run(this=0x0000000101b08a10) + 28 at thread.hpp:116
    frame #8: 0x000000010007f804 libboost_thread-mt.dylib`boost::(anonymous namespace)::thread_proxy(void*) + 164
    frame #9: 0x00007fff8e67199d libsystem_pthread.dylib`_pthread_body + 131
    frame #10: 0x00007fff8e67191a libsystem_pthread.dylib`_pthread_start + 168
    frame #11: 0x00007fff8e66f351 libsystem_pthread.dylib`thread_start + 13

  thread #5: tid = 0x8dd631, 0x00007fff86781db6 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #0: 0x00007fff86781db6 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #1: 0x00007fff8e672728 libsystem_pthread.dylib`_pthread_cond_wait + 767
    frame #2: 0x00000001010c9208 libbase.dylib`boost::condition_variable::wait(this=0x00000001011a1710, m=0x0000700000209a68) + 152 at condition_variable.hpp:77
    frame #3: 0x00000001010f2cc1 libbase.dylib`icinga::ThreadPool::WorkerThread::ThreadProc(this=0x00000001011a1838, queue=0x00000001011a16d0) + 1873 at threadpool.cpp:111
    frame #4: 0x00000001010fbaa1 libbase.dylib`boost::_mfi::mf1::operator(this=0x0000000101b08e80, t=0x00000001011a1838, a1=0x00000001011a16d0)(icinga::ThreadPool::WorkerThread&, icinga::ThreadPool::Queue&) const + 113 at mem_fn_template.hpp:186
    frame #5: 0x00000001010fb9fc libbase.dylib`void boost::_bi::list2, boost::reference_wrapper >::operator(this=0x0000000101b08e90, (null)=type @ 0x0000700000209e48, f=0x0000000101b08e80, a=0x0000700000209e70, (null)=0), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) + 108 at bind.hpp:319
    frame #6: 0x00000001010fb98a libbase.dylib`boost::_bi::bind_t, boost::_bi::list2, boost::reference_wrapper > >::operator(this=0x0000000101b08e80)() + 58 at bind.hpp:1222
    frame #7: 0x00000001010fb91c libbase.dylib`boost::detail::thread_data, boost::_bi::list2, boost::reference_wrapper > > >::run(this=0x0000000101b08c80) + 28 at thread.hpp:116
    frame #8: 0x000000010007f804 libboost_thread-mt.dylib`boost::(anonymous namespace)::thread_proxy(void*) + 164
    frame #9: 0x00007fff8e67199d libsystem_pthread.dylib`_pthread_body + 131
    frame #10: 0x00007fff8e67191a libsystem_pthread.dylib`_pthread_start + 168
    frame #11: 0x00007fff8e66f351 libsystem_pthread.dylib`thread_start + 13

  thread #6: tid = 0x8dd632, 0x00007fff86781db6 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #0: 0x00007fff86781db6 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #1: 0x00007fff8e672728 libsystem_pthread.dylib`_pthread_cond_wait + 767
    frame #2: 0x00000001010bbc23 libbase.dylib`boost::condition_variable::do_wait_until(this=0x00000001011a0c20, m=0x000070000028cac0, timeout=0x000070000028c5f8) + 131 at condition_variable.hpp:109
    frame #3: 0x00000001010b7dcc libbase.dylib`boost::condition_variable::timed_wait(this=0x00000001011a0c20, m=0x000070000028cac0, abs_time=0x000070000028c648) + 76 at condition_variable_fwd.hpp:127
    frame #4: 0x00000001010c9386 libbase.dylib`bool boost::condition_variable::timed_wait >(this=0x00000001011a0c20, m=0x000070000028cac0, wait_duration=0x000070000028cab8) + 166 at condition_variable_fwd.hpp:151
    frame #5: 0x00000001010f1b04 libbase.dylib`icinga::ThreadPool::ManagerThreadProc(this=0x00000001011a0a10) + 1492 at threadpool.cpp:240
    frame #6: 0x00000001010f6eb9 libbase.dylib`boost::_mfi::mf0::operator(this=0x0000000101b090a0, p=0x00000001011a0a10)(icinga::ThreadPool*) const + 105 at mem_fn_template.hpp:49
    frame #7: 0x00000001010f6e20 libbase.dylib`void boost::_bi::list1 >::operator(this=0x0000000101b090b0, (null)=type @ 0x000070000028ce48, f=0x0000000101b090a0, a=0x000070000028ce70, (null)=0), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) + 80 at bind.hpp:259
    frame #8: 0x00000001010f6dca libbase.dylib`boost::_bi::bind_t, boost::_bi::list1 > >::operator(this=0x0000000101b090a0)() + 58 at bind.hpp:1222
    frame #9: 0x00000001010f6d5c libbase.dylib`boost::detail::thread_data, boost::_bi::list1 > > >::run(this=0x0000000101b08ea0) + 28 at thread.hpp:116
    frame #10: 0x000000010007f804 libboost_thread-mt.dylib`boost::(anonymous namespace)::thread_proxy(void*) + 164
    frame #11: 0x00007fff8e67199d libsystem_pthread.dylib`_pthread_body + 131
    frame #12: 0x00007fff8e67191a libsystem_pthread.dylib`_pthread_start + 168
    frame #13: 0x00007fff8e66f351 libsystem_pthread.dylib`thread_start + 13

  thread #7: tid = 0x8dd633, 0x00007fff86781db6 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #0: 0x00007fff86781db6 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #1: 0x00007fff8e672728 libsystem_pthread.dylib`_pthread_cond_wait + 767
    frame #2: 0x00000001010bbc23 libbase.dylib`boost::condition_variable::do_wait_until(this=0x00000001011a25f0, m=0x000070000030fe48, timeout=0x000070000030fce8) + 131 at condition_variable.hpp:109
    frame #3: 0x00000001010b7dcc libbase.dylib`boost::condition_variable::timed_wait(this=0x00000001011a25f0, m=0x000070000030fe48, abs_time=0x000070000030fd38) + 76 at condition_variable_fwd.hpp:127
    frame #4: 0x00000001010c9386 libbase.dylib`bool boost::condition_variable::timed_wait >(this=0x00000001011a25f0, m=0x000070000030fe48, wait_duration=0x000070000030fe18) + 166 at condition_variable_fwd.hpp:151
    frame #5: 0x00000001010fc50c libbase.dylib`icinga::Timer::TimerThreadProc() + 460 at timer.cpp:270
    frame #6: 0x00000001010ffd36 libbase.dylib`boost::detail::thread_data::run(this=0x0000000101b090c0) + 22 at thread.hpp:116
    frame #7: 0x000000010007f804 libboost_thread-mt.dylib`boost::(anonymous namespace)::thread_proxy(void*) + 164
    frame #8: 0x00007fff8e67199d libsystem_pthread.dylib`_pthread_body + 131
    frame #9: 0x00007fff8e67191a libsystem_pthread.dylib`_pthread_start + 168
    frame #10: 0x00007fff8e66f351 libsystem_pthread.dylib`thread_start + 13

  thread #8: tid = 0x8dd66f, 0x00007fff86781db6 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #0: 0x00007fff86781db6 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #1: 0x00007fff8e672728 libsystem_pthread.dylib`_pthread_cond_wait + 767
    frame #2: 0x0000000104e95743 libchecker.dylib`boost::condition_variable::do_wait_until(this=0x00000001060ba738, m=0x000070000059ed80, timeout=0x000070000059e2c8) + 131 at condition_variable.hpp:109
    frame #3: 0x0000000104e9538c libchecker.dylib`boost::condition_variable::timed_wait(this=0x00000001060ba738, m=0x000070000059ed80, abs_time=0x000070000059e318) + 76 at condition_variable_fwd.hpp:127
    frame #4: 0x0000000104e31316 libchecker.dylib`bool boost::condition_variable::timed_wait >(this=0x00000001060ba738, m=0x000070000059ed80, wait_duration=0x000070000059ed48) + 166 at condition_variable_fwd.hpp:151
    frame #5: 0x0000000104e2c668 libchecker.dylib`icinga::CheckerComponent::CheckThreadProc(this=0x00000001060ba610) + 696 at checkercomponent.cpp:129
    frame #6: 0x0000000104e88d39 libchecker.dylib`boost::_mfi::mf0::operator(this=0x0000000101e97740, p=0x00000001060ba610)(icinga::CheckerComponent*) const + 105 at mem_fn_template.hpp:49
    frame #7: 0x0000000104e88c90 libchecker.dylib`void boost::_bi::list1 >::operator(this=0x0000000101e97750, (null)=type @ 0x000070000059ee48, f=0x0000000101e97740, a=0x000070000059ee70, (null)=0), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) + 80 at bind.hpp:259
    frame #8: 0x0000000104e88c1a libchecker.dylib`boost::_bi::bind_t, boost::_bi::list1 > >::operator(this=0x0000000101e97740)() + 58 at bind.hpp:1222
    frame #9: 0x0000000104e8888c libchecker.dylib`boost::detail::thread_data, boost::_bi::list1 > > >::run(this=0x0000000101e97540) + 28 at thread.hpp:116
    frame #10: 0x000000010007f804 libboost_thread-mt.dylib`boost::(anonymous namespace)::thread_proxy(void*) + 164
    frame #11: 0x00007fff8e67199d libsystem_pthread.dylib`_pthread_body + 131
    frame #12: 0x00007fff8e67191a libsystem_pthread.dylib`_pthread_start + 168
    frame #13: 0x00007fff8e66f351 libsystem_pthread.dylib`thread_start + 13

  thread #9: tid = 0x8dd670, 0x00007fff867832a2 libsystem_kernel.dylib`poll + 10
    frame #0: 0x00007fff867832a2 libsystem_kernel.dylib`poll + 10
    frame #1: 0x00000001010b5a3c libbase.dylib`icinga::Socket::Poll(this=0x0000000101f905e0, read=true, write=false, timeout=0x0000000000000000) + 220 at socket.cpp:394
    frame #2: 0x0000000105104004 libcompat.dylib`icinga::ExternalCommandListener::CommandPipeThread(this=0x0000000101e61c60, commandPath=0x0000000101fac458) + 2468 at externalcommandlistener.cpp:110
    frame #3: 0x00000001051153d1 libcompat.dylib`boost::_mfi::mf1::operator(this=0x0000000101fac440, p=0x0000000101e61c60, a1=0x0000000101fac458)(icinga::ExternalCommandListener*, icinga::String const&) const + 113 at mem_fn_template.hpp:165
    frame #4: 0x000000010511531f libcompat.dylib`void boost::_bi::list2, boost::_bi::value >::operator(this=0x0000000101fac450, (null)=type @ 0x0000700000621e48, f=0x0000000101fac440, a=0x0000700000621e70, (null)=0), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int) + 111 at bind.hpp:319
    frame #5: 0x000000010511528a libcompat.dylib`boost::_bi::bind_t, boost::_bi::list2, boost::_bi::value > >::operator(this=0x0000000101fac440)() + 58 at bind.hpp:1222
    frame #6: 0x000000010511497c libcompat.dylib`boost::detail::thread_data, boost::_bi::list2, boost::_bi::value > > >::run(this=0x0000000101fac240) + 28 at thread.hpp:116
    frame #7: 0x000000010007f804 libboost_thread-mt.dylib`boost::(anonymous namespace)::thread_proxy(void*) + 164
    frame #8: 0x00007fff8e67199d libsystem_pthread.dylib`_pthread_body + 131
    frame #9: 0x00007fff8e67191a libsystem_pthread.dylib`_pthread_start + 168
    frame #10: 0x00007fff8e66f351 libsystem_pthread.dylib`thread_start + 13

  thread #10: tid = 0x8dd671, 0x00007fff867813ca libsystem_kernel.dylib`__accept + 10
    frame #0: 0x00007fff867813ca libsystem_kernel.dylib`__accept + 10
    frame #1: 0x00000001010b55e2 libbase.dylib`icinga::Socket::Accept(this=0x0000000101e9a010) + 98 at socket.cpp:337
    frame #2: 0x000000010063bb8a libremote.dylib`icinga::ApiListener::ListenerThreadProc(this=0x0000000102802000, server=0x00007000006a4e20) + 138 at apilistener.cpp:223
    frame #3: 0x0000000100690061 libremote.dylib`boost::_mfi::mf1 const&>::operator(this=0x0000000101e9a710, p=0x0000000102802000, a1=0x00007000006a4e20)(icinga::ApiListener*, boost::intrusive_ptr const&) const + 113 at mem_fn_template.hpp:165
    frame #4: 0x000000010068ff8c libremote.dylib`void boost::_bi::list2, boost::_bi::value > >::operator(this=0x0000000101e9a720, (null)=type @ 0x00007000006a4e48, f=0x0000000101e9a710, a=0x00007000006a4e70, (null)=0) const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 const&>&, boost::_bi::list0&, int) + 124 at bind.hpp:319
    frame #5: 0x000000010068feea libremote.dylib`boost::_bi::bind_t const&>, boost::_bi::list2, boost::_bi::value > > >::operator(this=0x0000000101e9a710)() + 58 at bind.hpp:1222
    frame #6: 0x000000010068f67c libremote.dylib`boost::detail::thread_data const&>, boost::_bi::list2, boost::_bi::value > > > >::run(this=0x0000000101e9a510) + 28 at thread.hpp:116
    frame #7: 0x000000010007f804 libboost_thread-mt.dylib`boost::(anonymous namespace)::thread_proxy(void*) + 164
    frame #8: 0x00007fff8e67199d libsystem_pthread.dylib`_pthread_body + 131
    frame #9: 0x00007fff8e67191a libsystem_pthread.dylib`_pthread_start + 168
    frame #10: 0x00007fff8e66f351 libsystem_pthread.dylib`thread_start + 13

  thread #11: tid = 0x8dd672, 0x00007fff867832a2 libsystem_kernel.dylib`poll + 10
    frame #0: 0x00007fff867832a2 libsystem_kernel.dylib`poll + 10
    frame #1: 0x00000001010b5a3c libbase.dylib`icinga::Socket::Poll(this=0x0000000106783310, read=true, write=false, timeout=0x0000700000727da8) + 220 at socket.cpp:394
    frame #2: 0x00000001061900a1 liblivestatus.dylib`icinga::LivestatusListener::ServerThreadProc(this=0x000000010675def0) + 129 at livestatuslistener.cpp:154
    frame #3: 0x00000001061c33e9 liblivestatus.dylib`boost::_mfi::mf0::operator(this=0x0000000106782f90, p=0x000000010675def0)(icinga::LivestatusListener*) const + 105 at mem_fn_template.hpp:49
    frame #4: 0x00000001061c3340 liblivestatus.dylib`void boost::_bi::list1 >::operator(this=0x0000000106782fa0, (null)=type @ 0x0000700000727e48, f=0x0000000106782f90, a=0x0000700000727e70, (null)=0), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) + 80 at bind.hpp:259
    frame #5: 0x00000001061c32ca liblivestatus.dylib`boost::_bi::bind_t, boost::_bi::list1 > >::operator(this=0x0000000106782f90)() + 58 at bind.hpp:1222
    frame #6: 0x00000001061c2c9c liblivestatus.dylib`boost::detail::thread_data, boost::_bi::list1 > > >::run(this=0x0000000106782d90) + 28 at thread.hpp:116
    frame #7: 0x000000010007f804 libboost_thread-mt.dylib`boost::(anonymous namespace)::thread_proxy(void*) + 164
    frame #8: 0x00007fff8e67199d libsystem_pthread.dylib`_pthread_body + 131
    frame #9: 0x00007fff8e67191a libsystem_pthread.dylib`_pthread_start + 168
    frame #10: 0x00007fff8e66f351 libsystem_pthread.dylib`thread_start + 13

  thread #12: tid = 0x8dd673, 0x00007fff86781db6 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #0: 0x00007fff86781db6 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #1: 0x00007fff8e672728 libsystem_pthread.dylib`_pthread_cond_wait + 767
    frame #2: 0x00000001010c9208 libbase.dylib`boost::condition_variable::wait(this=0x0000000102802280, m=0x00007000007aa7f8) + 152 at condition_variable.hpp:77
    frame #3: 0x0000000101140586 libbase.dylib`icinga::WorkQueue::WorkerThreadProc(this=0x0000000102802218) + 1654 at workqueue.cpp:212
    frame #4: 0x00000001011439d9 libbase.dylib`boost::_mfi::mf0::operator(this=0x0000000101e9b730, p=0x0000000102802218)(icinga::WorkQueue*) const + 105 at mem_fn_template.hpp:49
    frame #5: 0x0000000101144320 libbase.dylib`void boost::_bi::list1 >::operator(this=0x0000000101e9b740, (null)=type @ 0x00007000007aae48, f=0x0000000101e9b730, a=0x00007000007aae70, (null)=0), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) + 80 at bind.hpp:259
    frame #6: 0x00000001011442ca libbase.dylib`boost::_bi::bind_t, boost::_bi::list1 > >::operator(this=0x0000000101e9b730)() + 58 at bind.hpp:1222
    frame #7: 0x000000010114425c libbase.dylib`boost::detail::thread_data, boost::_bi::list1 > > >::run(this=0x0000000101e9b530) + 28 at thread.hpp:116
    frame #8: 0x000000010007f804 libboost_thread-mt.dylib`boost::(anonymous namespace)::thread_proxy(void*) + 164
    frame #9: 0x00007fff8e67199d libsystem_pthread.dylib`_pthread_body + 131
    frame #10: 0x00007fff8e67191a libsystem_pthread.dylib`_pthread_start + 168
    frame #11: 0x00007fff8e66f351 libsystem_pthread.dylib`thread_start + 13

  thread #13: tid = 0x8dd676, 0x0000000100f16520 libbase.dylib`boost::intrusive_ptr::~intrusive_ptr(this=0x0000000103002c00) at intrusive_ptr.hpp:96
    frame #0: 0x0000000100f16520 libbase.dylib`boost::intrusive_ptr::~intrusive_ptr(this=0x0000000103002c00) at intrusive_ptr.hpp:96
    frame #1: 0x0000000100fa3378 libbase.dylib`icinga::ConfigObject::GetExtension(this=0x0000000103002c00, key=0x0000700000391920) + 184 at configobject.cpp:89
    frame #2: 0x0000000105b270a3 libdb_ido.dylib`icinga::DbObject::GetOrCreateByObject(object=0x0000700000391d30) + 131 at dbobject.cpp:291
    frame #3: 0x00000001054c477d libdb_ido_mysql.dylib`icinga::IdoMysqlConnection::FieldToEscapedString(this=0x0000000103809200, key=0x0000700000392360, value=0x0000700000392378, result=0x0000700000392398) + 509 at idomysqlconnection.cpp:741
    frame #4: 0x00000001054c914b libdb_ido_mysql.dylib`icinga::IdoMysqlConnection::CanExecuteQuery(this=0x0000000103809200, query=0x00000001067c6b10) + 1083 at idomysqlconnection.cpp:820
    frame #5: 0x00000001054c89a5 libdb_ido_mysql.dylib`icinga::IdoMysqlConnection::InternalExecuteMultipleQueries(this=0x0000000103809200, queries=0x00000001067a6248) + 453 at idomysqlconnection.cpp:852
    frame #6: 0x0000000105531411 libdb_ido_mysql.dylib`boost::_mfi::mf1 > const&>::operator(this=0x00000001067a6230, p=0x0000000103809200, a1=0x00000001067a6248)(icinga::IdoMysqlConnection*, std::__1::vector > const&) const + 113 at mem_fn_template.hpp:165
    frame #7: 0x000000010553136f libdb_ido_mysql.dylib`void boost::_bi::list2, boost::_bi::value > > >::operator(this=0x00000001067a6240, (null)=type @ 0x0000700000392598, f=0x00000001067a6230, a=0x00007000003925c0, (null)=0) > const&>, boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 > const&>&, boost::_bi::list0&, int) + 111 at bind.hpp:319
    frame #8: 0x00000001055312fa libdb_ido_mysql.dylib`boost::_bi::bind_t > const&>, boost::_bi::list2, boost::_bi::value > > > >::operator(this=0x00000001067a6230)() + 58 at bind.hpp:1222
    frame #9: 0x0000000105531060 libdb_ido_mysql.dylib`boost::detail::function::void_function_obj_invoker0 > const&>, boost::_bi::list2, boost::_bi::value > > > >, void>::invoke(function_obj_ptr=0x00007000003927d8) + 32 at function_template.hpp:159
    frame #10: 0x0000000100f370e8 libbase.dylib`boost::function0::operator(this=0x00007000003927d0)() const + 120 at function_template.hpp:770
    frame #11: 0x0000000101140c34 libbase.dylib`icinga::WorkQueue::WorkerThreadProc(this=0x0000000103809598) + 3364 at workqueue.cpp:228
    frame #12: 0x00000001011439d9 libbase.dylib`boost::_mfi::mf0::operator(this=0x0000000101fc02f0, p=0x0000000103809598)(icinga::WorkQueue*) const + 105 at mem_fn_template.hpp:49
    frame #13: 0x0000000101144320 libbase.dylib`void boost::_bi::list1 >::operator(this=0x0000000101fc0300, (null)=type @ 0x0000700000392e48, f=0x0000000101fc02f0, a=0x0000700000392e70, (null)=0), boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int) + 80 at bind.hpp:259
    frame #14: 0x00000001011442ca libbase.dylib`boost::_bi::bind_t, boost::_bi::list1 > >::operator(this=0x0000000101fc02f0)() + 58 at bind.hpp:1222
    frame #15: 0x000000010114425c libbase.dylib`boost::detail::thread_data, boost::_bi::list1 > > >::run(this=0x0000000101fc00f0) + 28 at thread.hpp:116
    frame #16: 0x000000010007f804 libboost_thread-mt.dylib`boost::(anonymous namespace)::thread_proxy(void*) + 164
    frame #17: 0x00007fff8e67199d libsystem_pthread.dylib`_pthread_body + 131
    frame #18: 0x00007fff8e67191a libsystem_pthread.dylib`_pthread_start + 168
    frame #19: 0x00007fff8e66f351 libsystem_pthread.dylib`thread_start + 13

  thread #14: tid = 0x8dd677, 0x00007fff86781db6 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #0: 0x00007fff86781db6 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #1: 0x00007fff8e672728 libsystem_pthread.dylib`_pthread_cond_wait + 767
    frame #2: 0x00000001010bbc23 libbase.dylib`boost::condition_variable::do_wait_until(this=0x00000001011a25f0, m=0x0000700000415e48, timeout=0x0000700000415ce8) + 131 at condition_variable.hpp:109
    frame #3: 0x00000001010b7dcc libbase.dylib`boost::condition_variable::timed_wait(this=0x00000001011a25f0, m=0x0000700000415e48, abs_time=0x0000700000415d38) + 76 at condition_variable_fwd.hpp:127
    frame #4: 0x00000001010c9386 libbase.dylib`bool boost::condition_variable::timed_wait >(this=0x00000001011a25f0, m=0x0000700000415e48, wait_duration=0x0000700000415e18) + 166 at condition_variable_fwd.hpp:151
    frame #5: 0x00000001010fc50c libbase.dylib`icinga::Timer::TimerThreadProc() + 460 at timer.cpp:270
    frame #6: 0x00000001010ffd36 libbase.dylib`boost::detail::thread_data::run(this=0x0000000106783630) + 22 at thread.hpp:116
    frame #7: 0x000000010007f804 libboost_thread-mt.dylib`boost::(anonymous namespace)::thread_proxy(void*) + 164
    frame #8: 0x00007fff8e67199d libsystem_pthread.dylib`_pthread_body + 131
    frame #9: 0x00007fff8e67191a libsystem_pthread.dylib`_pthread_start + 168
    frame #10: 0x00007fff8e66f351 libsystem_pthread.dylib`thread_start + 13

  thread #15: tid = 0x8dd67d, 0x00007fff867832a2 libsystem_kernel.dylib`poll + 10
    frame #0: 0x00007fff867832a2 libsystem_kernel.dylib`poll + 10
    frame #1: 0x000000010108c33b libbase.dylib`icinga::Process::IOThreadProc(tid=0) + 1707 at process.cpp:244
    frame #2: 0x000000010109286f libbase.dylib`void boost::_bi::list1 >::operator(this=0x0000000106817678, (null)=type @ 0x0000700000498e48, f=0x0000000106817670, a=0x0000700000498e70, (null)=0)(boost::_bi::type, void (*&)(int), boost::_bi::list0&, int) + 79 at bind.hpp:259
    frame #3: 0x000000010109281a libbase.dylib`boost::_bi::bind_t > >::operator(this=0x0000000106817670)() + 58 at bind.hpp:1222
    frame #4: 0x00000001010927ac libbase.dylib`boost::detail::thread_data > > >::run(this=0x0000000106817470) + 28 at thread.hpp:116
    frame #5: 0x000000010007f804 libboost_thread-mt.dylib`boost::(anonymous namespace)::thread_proxy(void*) + 164
    frame #6: 0x00007fff8e67199d libsystem_pthread.dylib`_pthread_body + 131
    frame #7: 0x00007fff8e67191a libsystem_pthread.dylib`_pthread_start + 168
    frame #8: 0x00007fff8e66f351 libsystem_pthread.dylib`thread_start + 13

  thread #16: tid = 0x8dd67e, 0x00007fff867832a2 libsystem_kernel.dylib`poll + 10
    frame #0: 0x00007fff867832a2 libsystem_kernel.dylib`poll + 10
    frame #1: 0x000000010108c33b libbase.dylib`icinga::Process::IOThreadProc(tid=1) + 1707 at process.cpp:244
    frame #2: 0x000000010109286f libbase.dylib`void boost::_bi::list1 >::operator(this=0x00000001067bdd28, (null)=type @ 0x000070000051be48, f=0x00000001067bdd20, a=0x000070000051be70, (null)=0)(boost::_bi::type, void (*&)(int), boost::_bi::list0&, int) + 79 at bind.hpp:259
    frame #3: 0x000000010109281a libbase.dylib`boost::_bi::bind_t > >::operator(this=0x00000001067bdd20)() + 58 at bind.hpp:1222
    frame #4: 0x00000001010927ac libbase.dylib`boost::detail::thread_data > > >::run(this=0x00000001067bdb20) + 28 at thread.hpp:116
    frame #5: 0x000000010007f804 libboost_thread-mt.dylib`boost::(anonymous namespace)::thread_proxy(void*) + 164
    frame #6: 0x00007fff8e67199d libsystem_pthread.dylib`_pthread_body + 131
    frame #7: 0x00007fff8e67191a libsystem_pthread.dylib`_pthread_start + 168
    frame #8: 0x00007fff8e66f351 libsystem_pthread.dylib`thread_start + 13

  thread #17: tid = 0x8dd67f, 0x00007fff867832a2 libsystem_kernel.dylib`poll + 10
    frame #0: 0x00007fff867832a2 libsystem_kernel.dylib`poll + 10
    frame #1: 0x000000010108c33b libbase.dylib`icinga::Process::IOThreadProc(tid=2) + 1707 at process.cpp:244
    frame #2: 0x000000010109286f libbase.dylib`void boost::_bi::list1 >::operator(this=0x000000010681a518, (null)=type @ 0x000070000082de48, f=0x000000010681a510, a=0x000070000082de70, (null)=0)(boost::_bi::type, void (*&)(int), boost::_bi::list0&, int) + 79 at bind.hpp:259
    frame #3: 0x000000010109281a libbase.dylib`boost::_bi::bind_t > >::operator(this=0x000000010681a510)() + 58 at bind.hpp:1222
    frame #4: 0x00000001010927ac libbase.dylib`boost::detail::thread_data > > >::run(this=0x000000010681a310) + 28 at thread.hpp:116
    frame #5: 0x000000010007f804 libboost_thread-mt.dylib`boost::(anonymous namespace)::thread_proxy(void*) + 164
    frame #6: 0x00007fff8e67199d libsystem_pthread.dylib`_pthread_body + 131
    frame #7: 0x00007fff8e67191a libsystem_pthread.dylib`_pthread_start + 168
    frame #8: 0x00007fff8e66f351 libsystem_pthread.dylib`thread_start + 13

  thread #18: tid = 0x8dd680, 0x00007fff867832a2 libsystem_kernel.dylib`poll + 10
    frame #0: 0x00007fff867832a2 libsystem_kernel.dylib`poll + 10
    frame #1: 0x000000010108c33b libbase.dylib`icinga::Process::IOThreadProc(tid=3) + 1707 at process.cpp:244
    frame #2: 0x000000010109286f libbase.dylib`void boost::_bi::list1 >::operator(this=0x00000001067b7838, (null)=type @ 0x00007000008b0e48, f=0x00000001067b7830, a=0x00007000008b0e70, (null)=0)(boost::_bi::type, void (*&)(int), boost::_bi::list0&, int) + 79 at bind.hpp:259
    frame #3: 0x000000010109281a libbase.dylib`boost::_bi::bind_t > >::operator(this=0x00000001067b7830)() + 58 at bind.hpp:1222
    frame #4: 0x00000001010927ac libbase.dylib`boost::detail::thread_data > > >::run(this=0x00000001067b7630) + 28 at thread.hpp:116
    frame #5: 0x000000010007f804 libboost_thread-mt.dylib`boost::(anonymous namespace)::thread_proxy(void*) + 164
    frame #6: 0x00007fff8e67199d libsystem_pthread.dylib`_pthread_body + 131
    frame #7: 0x00007fff8e67191a libsystem_pthread.dylib`_pthread_start + 168
    frame #8: 0x00007fff8e66f351 libsystem_pthread.dylib`thread_start + 13

icinga-migration · 2016-07-12T10:00:07Z

Updated by gbeutner on 2016-07-12 10:00:07 +00:00

Status changed from Assigned to Resolved
Done % changed from 0 to 100

Applied in changeset ea57f00.

icinga-migration · 2016-07-12T10:03:56Z

Updated by mfriedrich on 2016-07-12 10:03:56 +00:00

-> wrong config for categories

icinga-migration closed this as completed Jul 12, 2016

icinga-migration added bug Something isn't working area/db-ido Database output labels Jan 17, 2017

icinga-migration added this to the 2.5.0 milestone Jan 17, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[dev.icinga.com #12127] DB IDO query queue does not clean up with v2.4.10-520-g124c80b #4356

[dev.icinga.com #12127] DB IDO query queue does not clean up with v2.4.10-520-g124c80b #4356

icinga-migration commented Jul 7, 2016

icinga-migration commented Jul 11, 2016

icinga-migration commented Jul 12, 2016

icinga-migration commented Jul 12, 2016

icinga-migration commented Jul 12, 2016

icinga-migration commented Jul 12, 2016

icinga-migration commented Jul 12, 2016