Should you run into problems with Icinga please add this file to your help request Started collection at 2016-03-02 10:01:36 ==================================================== ============== GENERAL INFORMATION ============== Application version: v2.4.3 Installation root: /usr Sysconf directory: /etc Run directory: /var/run Local state directory: /var Package data directory: /usr/share/icinga2 State path: /var/lib/icinga2/icinga2.state Objects path: /var/cache/icinga2/icinga2.debug Vars path: /var/cache/icinga2/icinga2.vars PID path: /var/run/icinga2/icinga2.pid Enabled features: api checker command graphite ido-pgsql mainlog notification Disabled features: compatlog debuglog gelf icingastatus livestatus opentsdb perfdata statusdata syslog ######################## debuglog is disabled, please activate it and rerun icinga2 ######################## ============== OBJECT INFORMATION ============== Checking object file from /var/cache/icinga2/icinga2.debug Found the 660 objects: Type : Count ApiListener : 1 ApiUser : 1 CheckCommand : 174 CheckerComponent : 1 Comment : 2 Dependency : 98 Endpoint : 5 ExternalCommandListener : 1 FileLogger : 1 GraphiteWriter : 1 Host : 5 HostGroup : 3 IcingaApplication : 1 IdoPgsqlConnection : 1 Notification : 174 NotificationCommand : 3 NotificationComponent : 1 Service : 170 ServiceGroup : 4 TimePeriod : 5 User : 3 UserGroup : 3 Zone : 2 The objects origins are: /etc/icinga2/conf.d/RIO-common/RIO-CommonServices.conf /etc/icinga2/conf.d/RIO-common/RIO-DBchecks.conf /etc/icinga2/conf.d/RIO-common/RIO-Dependencies.conf /etc/icinga2/conf.d/RIO-common/RIO-Groups.conf /etc/icinga2/conf.d/RIO-common/RIO-SAPCentralServices.conf /etc/icinga2/conf.d/RIO-common/RIO-abap-services.conf /etc/icinga2/conf.d/RIO-common/RIO-checkcommands.conf /etc/icinga2/conf.d/RIO-common/RIO-disks.conf /etc/icinga2/conf.d/RIO-common/RIO-j2ee-services.conf /etc/icinga2/conf.d/RIO-common/RIO-notification.conf /etc/icinga2/conf.d/RIO-common/RIO-timeperiods.conf /etc/icinga2/conf.d/RIO-common/RIO-user.conf /etc/icinga2/conf.d/api-users.conf /etc/icinga2/conf.d/app.conf /etc/icinga2/conf.d/commands.conf /etc/icinga2/conf.d/customer/AAA/AAA-hosts.conf /etc/icinga2/conf.d/customer/BBB/BBB-hosts.conf /etc/icinga2/conf.d/customer/RIO/RIO-commmands.conf /etc/icinga2/conf.d/customer/RIO/RIO-hosts.conf /etc/icinga2/conf.d/customer/RIO/RIO-notification.conf /etc/icinga2/conf.d/customer/RIO/RIO-users.conf /etc/icinga2/conf.d/groups.conf /etc/icinga2/conf.d/hosts.conf /etc/icinga2/conf.d/services.conf /etc/icinga2/conf.d/timeperiods.conf /etc/icinga2/conf.d/users.conf /etc/icinga2/features-enabled/api.conf /etc/icinga2/features-enabled/checker.conf /etc/icinga2/features-enabled/command.conf /etc/icinga2/features-enabled/graphite.conf /etc/icinga2/features-enabled/ido-pgsql.conf /etc/icinga2/features-enabled/mainlog.conf /etc/icinga2/features-enabled/notification.conf /etc/icinga2/zones.conf /usr/share/icinga2/include/command-icinga.conf /usr/share/icinga2/include/command-plugins-windows.conf /usr/share/icinga2/include/command-plugins.conf /usr/share/icinga2/include/plugins-contrib.d/databases.conf /usr/share/icinga2/include/plugins-contrib.d/hardware.conf /usr/share/icinga2/include/plugins-contrib.d/ipmi.conf /usr/share/icinga2/include/plugins-contrib.d/network-components.conf /usr/share/icinga2/include/plugins-contrib.d/operating-system.conf /usr/share/icinga2/include/plugins-contrib.d/virtualization.conf /usr/share/icinga2/include/plugins-contrib.d/vmware.conf /usr/share/icinga2/include/plugins-contrib.d/web.conf /var/lib/icinga2/api/packages/_api/ih1.ghi.net-1450178447-1/conf.d/comments/sl1.ads.sha.net!Memory!ih1.ghi.net-1455280978-0.conf /var/lib/icinga2/api/packages/_api/ih1.ghi.net-1450178447-1/conf.d/comments/sl1.ads.sha.net!SAP J2EE http port SL1-00!ih1.ghi.net-1455280978-4.conf db_ido-itl.conf ============== LOGS AND CRASH REPORTS ============== Getting the last 20 lines of 1 FileLogger objects. Logger main-log at path: /var/log/icinga2/icinga2.log [begin: '/var/log/icinga2/icinga2.log' line: 300126] # (5) libcompat.so: icinga::ExternalCommandListener::CommandPipeThread(icinga::String const&) (+0x299) [0x7fc01f84a5e9] # (6) /usr/lib64/libboost_thread.so.1.53.0() [0x3e6860c5c3] # (7) /lib64/libpthread.so.0() [0x371c807a51] # (8) libc.so.6: clone (+0x6d) [0x371c4e893d] # # # [2016-03-02 10:01:35 +0100] information/ExternalCommandListener: Executing external command: [1456909295] PROCESS_SERVICE_CHECK_RESULT;host2.abc.net;SAP_ENQ-SM1-00;0;Usage Enqueue table: 5 - OK|SapEnqTab=5;50;75;0; # [2016-03-02 10:01:35 +0100] information/ExternalCommandListener: Executing external command: [1456909295] PROCESS_SERVICE_CHECK_RESULT;host2.abc.net;SAP_ENQ_OLD-SM1-00;0;old Enqueue entries: 2 - OK|SapEnqOld=2;5;500;0; # [2016-03-02 10:01:35 +0100] information/ExternalCommandListener: Executing external command: [1456909295] PROCESS_SERVICE_CHECK_RESULT;host2.abc.net;SAP_DD_req_abort-SM1-00;0;DD aborted requests: 0 - OK|SapDdReqAbort=0;1;10;0; # [2016-03-02 10:01:35 +0100] information/ExternalCommandListener: Executing external command: [1456909295] PROCESS_SERVICE_CHECK_RESULT;host2.abc.net;SAP_Spool_Reorg-SM1-00;0;Spool reorg: 1 - OK|SapSpoolReorg=1;0;0;0; # [2016-03-02 10:01:35 +0100] information/ExternalCommandListener: Executing external command: [1456909295] PROCESS_SERVICE_CHECK_RESULT;host2.abc.net;SAP_ABAP_Memory-SM1-00;0;SAP ABAP mem. - OK| PxaAllocSize=1024000000b;0;0 PxaFree=47041536b;0;0 PxaSwap=0b;0;0 RollShmArea=268435456b;0;0 RollFile=1779564544b;0;0 RollCurr=12283904b;0;0 RollMax=43048960b;0;0 PagingShm=268435456b;0;0 PagingFile=1779564544b;0;0 PagingCurr=88064b;0;0 PagingMax=99442688b;0;0 EmTotal=4290772992b;0;0 Em100Used=21;0;0 EmMax=1694498816b;0;0 # [2016-03-02 10:01:35 +0100] information/ExternalCommandListener: Executing external command: [1456909295] PROCESS_SERVICE_CHECK_RESULT;host2.abc.net;SAP_Syst_changeable-SM1-00;2;System change option: critical # [2016-03-02 10:01:35 +0100] information/ExternalCommandListener: Executing external command: [1456909295] PROCESS_SERVICE_CHECK_RESULT;host2.abc.net;SAP_P_ClientSetting-SM1-00;2;No P-client maintained: critical # [2016-03-02 10:01:35 +0100] information/ExternalCommandListener: Executing external command: [1456909295] PROCESS_SERVICE_CHECK_RESULT;host2.abc.net;SAP_Crit_User_Passw-SM1-00;0;Critical User Password: OK # [2016-03-02 10:01:35 +0100] information/ExternalCommandListener: Executing external command: [1456909295] PROCESS_SERVICE_CHECK_RESULT;host2.abc.net;SAP_Cert-SM1-00;0;Certificate duration: OK / sm1, SM1, SYS, List, CN=SM1, 20330701, 6330 days / sm1, SM1, SSLS, Own, CN=host2.abc.net, 20380101, 7975 days / sm1, SM1, SAP_AGS_OLCNT_VERIFY, Own, CN=SM1, 20380101, 7975 days / sm1, SM1, SAP_AGS_OLCNT_VERIFY, List, CN=Online Recommendations, 20380101, 7975 days / sm1, SM1, SAP_AGS_OLCNT_VERIFY, List, CN=Online Recommendations Upd, 20380101, 7975 days / sm1, SM1, SYS, Own, CN=SM1, 20380101, 7975 days # [2016-03-02 10:01:35 +0100] information/ExternalCommandListener: Executing external command: [1456909295] PROCESS_SERVICE_CHECK_RESULT;host2.abc.net;SAP_RFC_Check-SM_P30VUBCLNT000_READ-SM1-00;0;Connection: OK # [2016-03-02 10:01:35 +0100] information/ExternalCommandListener: Executing external command: [1456909295] PROCESS_SERVICE_CHECK_RESULT;host2.abc.net;SAP_RFC_Check-TESTCONN-SM1-00;2;Connection Error: critical # [2016-03-02 10:01:36 +0100] information/JsonRpcConnection: Reconnecting to API endpoint 'iaz.ihle.eusap.internal' via host '10.79.1.19' and port '5665' # [2016-03-02 10:01:36 +0100] critical/TcpSocket: Invalid socket: Connection refused # [2016-03-02 10:01:36 +0100] critical/ApiListener: Cannot connect to host '10.30.10.10' on port '5665' [end: '/var/log/icinga2/icinga2.log' line: 300146] Latest crash report is from 2016-03-02 10:01:37 File: /var/log/icinga2/crash/report.1456215785.3597901 [begin: '/var/log/icinga2/crash/report.1456215785.3597901'] [end: '/var/log/icinga2/crash/report.1456215785.3597901'] ============== CONFIGURATION FILES ============== A collection of important configuration files follows, please make sure to remove any sensitive data such as credentials, internal company names, etc [begin: '/etc/icinga2/icinga2.conf'] # /** # * Icinga 2 configuration file # * - this is where you define settings for the Icinga application including # * which hosts/services to check. # * # * For an overview of all available configuration options please refer # * to the documentation that is distributed as part of Icinga 2. # */ # # /** # * The constants.conf defines global constants. # */ # include "constants.conf" # # /** # * The zones.conf defines zones for a cluster setup. # * Not required for single instance setups. # */ # include "zones.conf" # # /** # * The Icinga Template Library (ITL) provides a number of useful templates # * and command definitions. # * Common monitoring plugin command definitions are included separately. # */ # include # include # include # include # # /** # * The features-available directory contains a number of configuration # * files for features which can be enabled and disabled using the # * icinga2 feature enable / icinga2 feature disable CLI commands. # * These commands work by creating and removing symbolic links in # * the features-enabled directory. # */ # include "features-enabled/*.conf" # # /** # * The repository.d directory contains all configuration objects # * managed by the 'icinga2 repository' CLI commands. # */ # include_recursive "repository.d" # # /** # * Although in theory you could define all your objects in this file # * the preferred way is to create separate directories and files in the conf.d # * directory. Each of these files must have the file extension ".conf". # */ # include_recursive "conf.d" [end: '/etc/icinga2/icinga2.conf'] [begin: '/etc/icinga2/zones.conf'] # /* # * Generated by Icinga 2 node setup commands # * on 2016-01-07 09:19:14 +0100 # */ # # object Endpoint NodeName { # } # # object Zone ZoneName { # endpoints = [ NodeName ]; # } # # object Endpoint "host1.abc.net" { # host = "10.10.10.10" # port = 5665 # } # # # RIO # object Endpoint "host2.abc.net" { # host = "10.10.10.11" # port = 5665 # } # # # # BBB # object Endpoint "host3.xyz.net" { # host = "10.20.10.10" # port = 5665 # } # # # AAA # object Endpoint "host4.edf.net" { # host = "10.30.10.10" # //port = 5665 # } # # # # remote clients # object Zone "remote-client" { # endpoints = [ "host1.abc.net", # "host2.abc.net", # "host3.xyz.net", # "host4.edf.net" ] # parent = "master1" # } [end: '/etc/icinga2/zones.conf'] ==================================================== Finished collection at 2016-03-02 10:01:37 Took 0.60400199890136719 seconds