[dev.icinga.com #9570] Multi select causes commands to fail

[icinga-migration]

This issue has been migrated from Redmine: https://dev.icinga.com/issues/9570

Created by smadmin on 2015-07-07 07:34:12 +00:00

Assignee: (none)
Status: Closed (closed on 2016-02-24 23:55:42 +00:00)
Target Version: (none)
Last Update: 2016-02-24 23:55:42 +00:00 (in Redmine)

Icinga Version: v2.3.8

---

Using multi select on hosts or service causes commands to fail with message

Can't send external Icinga command to the local command file "/usr/local/nagios/icinga2.cmd": No such device or address


#0 /usr/share/icingaweb2/modules/monitoring/application/forms/Command/Object/CheckNowCommandForm.php(68): Icinga\Module\Monitoring\Command\Transport\LocalCommandFile->send(Object(Icinga\Module\Monitoring\Command\Object\ScheduleServiceCheckCommand))
#1 /usr/share/php/Icinga/Web/Form.php(909): Icinga\Module\Monitoring\Forms\Command\Object\CheckNowCommandForm->onSuccess()
#2 /usr/share/icingaweb2/modules/monitoring/application/controllers/ServicesController.php(98): Icinga\Web\Form->handleRequest()
#3 /usr/share/icingaweb2/library/vendor/Zend/Controller/Action.php(507): Monitoring_ServicesController->showAction()
#4 /usr/share/icingaweb2/library/vendor/Zend/Controller/Dispatcher/Standard.php(303): Zend_Controller_Action->dispatch('showAction')
#5 /usr/share/icingaweb2/library/vendor/Zend/Controller/Front.php(937): Zend_Controller_Dispatcher_Standard->dispatch(Object(Icinga\Web\Request), Object(Icinga\Web\Response))
#6 /usr/share/php/Icinga/Application/Web.php(154): Zend_Controller_Front->dispatch(Object(Icinga\Web\Request), Object(Icinga\Web\Response))
#7 /usr/share/php/Icinga/Application/webrouter.php(111): Icinga\Application\Web->dispatch()
#8 /usr/share/icingaweb2/public/index.php(4): require_once('/usr/share/php/...')
#9 {main}

while executed on a single object it works.

Changesets

2015-08-10 09:31:27 +00:00 by elippmann cb0b3c8754b0e8b1dc96b240f5f40a084b0503c2

monitoring: Let PHP flush the writer buffer to the command file

refs #9570

---

Relations:

• duplicates #9570

[icinga-migration]

Updated by jmeyer on 2015-07-07 07:53:35 +00:00

• Status changed from New to Feedback
• Assigned to set to smadmin

Hi,

which version of Icinga Web 2 are you using?
This should have been fixed long ago: #8815 (I guess you're using a remote instance)

Best regards,
Johannes

[icinga-migration]

Updated by dgoetz on 2015-07-07 08:20:00 +00:00

• Assigned to deleted smadmin

Using the release candidate and a local command pipe

[icinga-migration]

Updated by jmeyer on 2015-07-07 08:37:55 +00:00

I cannot reproduce this, neither with rc1 nor the latest master. Please make sure you've upgraded properly. (all files are up2date etc)

If the issue persists:

• Enable debug log
• Submit a single command
• Submit multiple commands
• Copy&Paste the respective log lines as well as your instances.ini (/etc/icingaweb2/modules/monitoring) here

[icinga-migration]

Updated by smadmin on 2015-07-07 09:39:55 +00:00

We see this in the log:

2015-07-07T11:21:49+02:00 - DEBUG - Sending external Icinga command "[1436260909] SCHEDULE_FORCED_SVC_CHECK;SERVER1;linux_daemon_nrpe;1436260909" to the local command file "/usr/local/nagios/icinga2.cmd"
2015-07-07T11:21:57+02:00 - DEBUG - Sending external Icinga command "[1436260917] SCHEDULE_FORCED_SVC_CHECK;SERVER1;linux_daemon_nrpe;1436260917" to the local command file "/usr/local/nagios/icinga2.cmd"
2015-07-07T11:21:57+02:00 - DEBUG - Sending external Icinga command "[1436260917] SCHEDULE_FORCED_SVC_CHECK;SERVER2;linux_daemon_nrpe;1436260917" to the local command file "/usr/local/nagios/icinga2.cmd"
2015-07-07T11:21:57+02:00 - ERROR - Icinga\Module\Monitoring\Command\Exception\TransportException in /usr/share/icingaweb2/modules/monitoring/library/Monitoring/Command/Transport/LocalCommandFile.php:132 with message: Can't send external Icinga command to the local command file "/usr/local/nagios/icinga2.cmd":  No such device or address
2015-07-07T11:21:57+02:00 - ERROR - Stacktrace: #0 /usr/share/icingaweb2/modules/monitoring/application/forms/Command/Object/CheckNowCommandForm.php(68): Icinga\Module\Monitoring\Command\Transport\LocalCommandFile->send(Object(Icinga\Module\Monitoring\Command\Object\ScheduleServiceCheckCommand))

#1 /usr/share/php/Icinga/Web/Form.php(909): Icinga\Module\Monitoring\Forms\Command\Object\CheckNowCommandForm->onSuccess()
#2 /usr/share/icingaweb2/modules/monitoring/application/controllers/ServicesController.php(98): Icinga\Web\Form->handleRequest()
#3 /usr/share/icingaweb2/library/vendor/Zend/Controller/Action.php(507): Monitoring_ServicesController->showAction()
#4 /usr/share/icingaweb2/library/vendor/Zend/Controller/Dispatcher/Standard.php(303): Zend_Controller_Action->dispatch('showAction')
#5 /usr/share/icingaweb2/library/vendor/Zend/Controller/Front.php(937): Zend_Controller_Dispatcher_Standard->dispatch(Object(Icinga\Web\Request), Object(Icinga\Web\Response))
#6 /usr/share/php/Icinga/Application/Web.php(154): Zend_Controller_Front->dispatch(Object(Icinga\Web\Request), Object(Icinga\Web\Response))
#7 /usr/share/php/Icinga/Application/webrouter.php(111): Icinga\Application\Web->dispatch()
#8 /usr/share/icingaweb2/public/index.php(4): require_once('/usr/share/php/...')
#9 {main}

First line is from the single command, afterwards from selecting 5 services. It seems to fail from the second service on, so perhaps the pipe is blocked. Directly inserting command in a while loop does not block it and all commands are issued.

# cat /etc/icingaweb2/modules/monitoring/instances.ini 
[icinga]
transport           = "local"
path                = "/usr/local/nagios/icinga2.cmd"

[icinga-migration]

Updated by jmeyer on 2015-07-07 11:08:03 +00:00

Please apply the following patch and report whether it fixes the issue:

diff --git a/modules/monitoring/application/forms/Command/Object/CheckNowCommandForm.php b/modules/monitoring/application/forms/Command/Object/CheckNowCommandForm.php
index 850443e..10c06e9 100644
--- a/modules/monitoring/application/forms/Command/Object/CheckNowCommandForm.php
+++ b/modules/monitoring/application/forms/Command/Object/CheckNowCommandForm.php
@@ -65,7 +65,11 @@ class CheckNowCommandForm extends ObjectsCommandForm
                 ->setObject($object)
                 ->setForced()
                 ->setCheckTime(time());
-            $this->getTransport($this->request)->send($check);
+            $transport = $this->getTransport($this->request);
+            if ($transport::TRANSPORT === 'local') {
+                $transport->setOpenMode('w');
+            }
+            $transport->send($check);
         }
         Notification::success(mtp(
             'monitoring',

[icinga-migration]

Updated by smadmin on 2015-07-07 13:00:55 +00:00

This removes the error message, but let it fail silently while sending if pipe is blocked. (I can only see one or two external commands in icinga2.log out of five executed in the webinterface).

[icinga-migration]

Updated by jmeyer on 2015-07-07 14:30:48 +00:00

Then there is something else wrong in your environment. Using fopen(, "w") is a blocking call, which will not succeed until something (Icinga 2) actually reads from the pipe. The patch I've proposed behaves the same as your while loop and issues one command at a time and continues after Icinga 2 retrieved the command.

I've even tried it myself to be really sure:

• Stopped Icinga 2
• Scheduled five host checks (Web page loads infinitely as the first write attempt blocks the process)
• Started Icinga 2
• All five commands were received and processed by Icinga 2

How did you install Icinga 2?
Which operating system are you running?

[icinga-migration]

Updated by vvv on 2015-07-07 22:29:30 +00:00

I was seeing similar issue in 3.beta3. 4.rc1 seems to have fixed it for me.

[icinga-migration]

Updated by elippmann on 2015-07-16 14:11:27 +00:00

• Status changed from Feedback to New
• Target Version set to 271

[icinga-migration]

Updated by elippmann on 2015-07-17 14:23:08 +00:00

• Estimated Hours set to 0.5

[icinga-migration]

Updated by icinga-kanban on 2015-08-10 09:34:47 +00:00

Build !#939 triggered by commit cb0b3c8 passed successfully.

Branch: origin/master
Author: Eric Lippmann

[icinga-migration]

Updated by jmeyer on 2015-08-18 06:38:29 +00:00

• Duplicated set to 9938

[icinga-migration]

Updated by jmeyer on 2015-08-18 09:12:59 +00:00

• Project changed from Icinga Web 2 to Icinga 2
• Category deleted 148
• Priority changed from Normal to High
• Target Version deleted 271
• Estimated Hours deleted 0.5
• Icinga Version set to v2.3.8

Hi Icinga2!

We suspect that this issue is caused by your commandpipe listener implementation. Can you please make your devs have a look at it if there is something to improve when processing multiple commands?

Thanks,
Johannes

[icinga-migration]

Updated by mfriedrich on 2015-08-18 09:19:13 +00:00

• Category set to libicinga
• Priority changed from High to Normal

[icinga-migration]

Updated by mfriedrich on 2016-02-24 23:55:42 +00:00

• Status changed from New to Closed

Haven't seen the issue after the release of 2.4.0. Please retest and open a new issue if it still occurs.