Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dev.icinga.com #11006] Segfault in ApiListener::ConfigUpdateObjectAPIHandler #3863

Closed
icinga-migration opened this issue Jan 21, 2016 · 10 comments
Labels
area/distributed Distributed monitoring (master, satellites, clients) bug Something isn't working
Milestone

Comments

@icinga-migration
Copy link

This issue has been migrated from Redmine: https://dev.icinga.com/issues/11006

Created by bsheqa on 2016-01-21 13:37:35 +00:00

Assignee: gbeutner
Status: Resolved (closed on 2016-02-09 08:14:33 +00:00)
Target Version: 2.4.2
Last Update: 2016-02-23 10:54:47 +00:00 (in Redmine)

Icinga Version: r2.4.1-1
Backport?: Already backported
Include in Changelog: 1

Application information:
  Application version: r2.4.1-1
  Installation root: /usr
  Sysconf directory: /etc
  Run directory: /run
  Local state directory: /var
  Package data directory: /usr/share/icinga2
  State path: /var/lib/icinga2/icinga2.state
  Modified attributes path: /var/lib/icinga2/modified-attributes.conf
  Objects path: /var/cache/icinga2/icinga2.debug
  Vars path: /var/cache/icinga2/icinga2.vars
  PID path: /run/icinga2/icinga2.pid

System information:
  Platform: _NAME="Ubuntu 14.04.3 LTS
  Platform version: ID="14.04
  Kernel: Linux
  Kernel version: 3.13.0-74-generic
  Architecture: x86_64
Stacktrace:

    (0) libpthread.so.0:  (+0x10340) [0x7f89eaef1340]
    (1) libc.so.6: gsignal (+0x39) [0x7f89e9c92cc9]
    (2) libc.so.6: abort (+0x148) [0x7f89e9c960d8]
    (3) libc.so.6:  (+0x2fb86) [0x7f89e9c8bb86]
    (4) libc.so.6:  (+0x2fc32) [0x7f89e9c8bc32]
    (5) libremote.so:  (+0xb0cb3) [0x7f89e80e1cb3]
    (6) libremote.so: icinga::ApiListener::ConfigUpdateObjectAPIHandler(boost::intrusive_ptr const&, boost::intrusive_ptr const&) (+0xe95) [0x7f89e811c385]
    (7) libremote.so: boost::detail::function::function_invoker2 const&, boost::intrusive_ptr const&), icinga::Value, boost::intrusive_ptr const&, boost::intrusive_ptr const&>::invoke(boost::detail::function::function_buffer&, boost::intrusive_ptr const&, boost::intrusive_ptr const&) (+0xf) [0x7f89e813acdf]
    (8) libremote.so: icinga::ApiFunction::Invoke(boost::intrusive_ptr const&, boost::intrusive_ptr const&) (+0x1d) [0x7f89e80eb68d]
    (9) libremote.so: icinga::JsonRpcConnection::ProcessMessage() (+0x524) [0x7f89e8118eb4]
    (10) libremote.so: icinga::JsonRpcConnection::DataAvailableHandler() (+0x38) [0x7f89e8135848]
    (11) libbase.so: boost::signals2::detail::signal_impl const&), boost::signals2::optional_last_value, int, std::less, boost::function const&)>, boost::function const&)>, boost::signals2::mutex>::operator()(boost::intrusive_ptr const&) (+0x1cc) [0x7f89eac67f2c]
    (12) libbase.so: icinga::Stream::SignalDataAvailable() (+0x30) [0x7f89eac16620]
    (13) libbase.so: icinga::TlsStream::OnEvent(int) (+0x3e8) [0x7f89eac16b48]
    (14) libbase.so: icinga::SocketEvents::ThreadProc() (+0x23a) [0x7f89eac136aa]
    (15) libboost_thread.so.1.54.0:  (+0xba4a) [0x7f89eb57ca4a]
    (16) libpthread.so.0:  (+0x8182) [0x7f89eaee9182]
    (17) libc.so.6: clone (+0x6d) [0x7f89e9d5647d]

***
* This would indicate a runtime problem or configuration error. If you believe this is a bug in Icinga 2
* please submit a bug report at https://dev.icinga.org/ and include this stack trace as well as any other
* information that might be useful in order to reproduce this problem.
***

Changesets

2016-02-09 08:13:43 +00:00 by (unknown) 14d805e

Fix crash in ApiListener::ConfigUpdateObjectAPIHandler

fixes #11006

2016-02-12 13:15:24 +00:00 by mfriedrich c8a9f45

ApiListener: Bail out if object could not be created during config sync

refs #11006

2016-02-23 08:44:26 +00:00 by (unknown) 2f948ca

Fix crash in ApiListener::ConfigUpdateObjectAPIHandler

fixes #11006

2016-02-23 08:44:26 +00:00 by mfriedrich a30b112

ApiListener: Bail out if object could not be created during config sync

refs #11006

Relations:

@icinga-migration
Copy link
Author

Updated by gbeutner on 2016-01-26 13:40:10 +00:00

  • Description updated

@icinga-migration
Copy link
Author

Updated by bsheqa on 2016-01-27 10:00:16 +00:00

[2016-01-27 10:52:43 +0100] warning/JsonRpcConnection: Error while reading JSON-RPC message for identity 'host1': Error: std::exception

        (0) libbase.so: void boost::throw_exception(icinga::openssl_error const&) (+0x97) [0x7ffff7286c67]
        (1) libbase.so: void boost::exception_detail::throw_exception_(icinga::openssl_error const&, char const*, char const*, int) (+0x40) [0x7ffff7286d10]
        (2) libbase.so: icinga::TlsStream::HandleError() const (+0xbc) [0x7ffff7223d2c]
        (3) libbase.so: icinga::TlsStream::Read(void*, unsigned long, bool) (+0x83) [0x7ffff7223eb3]
        (4) libbase.so: icinga::StreamReadContext::FillFromStream(boost::intrusive_ptr const&, bool) (+0x7c) [0x7ffff72339ac]
        (5) libbase.so: icinga::NetString::ReadStringFromStream(boost::intrusive_ptr const&, icinga::String*, icinga::StreamReadContext&, bool) (+0xce) [0x7ffff72420ce]
        (6) libremote.so: icinga::JsonRpc::ReadMessage(boost::intrusive_ptr const&, boost::intrusive_ptr*, icinga::StreamReadContext&, bool) (+0x3d) [0x7ffff474acad]
        (7) libremote.so: icinga::JsonRpcConnection::ProcessMessage() (+0x65) [0x7ffff476b9f5]
        (8) libremote.so: icinga::JsonRpcConnection::DataAvailableHandler() (+0x38) [0x7ffff4788848]
        (9) libbase.so: boost::signals2::detail::signal_impl const&), boost::signals2::optional_last_value, int, std::less, boost::function const&)>, boost::function const&)>, boost::signals2::mutex>::operator()(boost::intrusive_ptr const&) (+0x1cc) [0x7ffff72baf2c]
        (10) libbase.so: icinga::Stream::SignalDataAvailable() (+0x30) [0x7ffff7269620]
        (11) libbase.so: icinga::TlsStream::OnEvent(int) (+0x3a8) [0x7ffff7269b08]
        (12) libbase.so: icinga::SocketEvents::ThreadProc() (+0x23a) [0x7ffff72666aa]
        (13) libboost_thread.so.1.54.0:  (+0xba4a) [0x7ffff7bcfa4a]
        (14) libpthread.so.0:  (+0x8182) [0x7ffff753c182]
        (15) libc.so.6: clone (+0x6d) [0x7ffff63a947d]


[2016-01-27 10:52:43 +0100] warning/JsonRpcConnection: API client disconnected for identity 'host1'

Program received signal SIGPIPE, Broken pipe.
[Switching to Thread 0x7fffe09d2700 (LWP 7227)]
0x00007ffff754335d in write () at ../sysdeps/unix/syscall-template.S:81
81      ../sysdeps/unix/syscall-template.S: No such file or directory.

(gdb) bt

#0  0x00007ffff754335d in write () at ../sysdeps/unix/syscall-template.S:81
#1  0x00007ffff56407f6 in ?? () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#2  0x00007ffff563e89b in BIO_write () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
#3  0x00007ffff595dec2 in ?? () from /lib/x86_64-linux-gnu/libssl.so.1.0.0
#4  0x00007ffff595faa3 in ?? () from /lib/x86_64-linux-gnu/libssl.so.1.0.0
#5  0x00007ffff595bf62 in ?? () from /lib/x86_64-linux-gnu/libssl.so.1.0.0
#6  0x00007ffff723358d in icinga::TlsStream::Close (this=0x7fff9c044070) at /build/icinga2-upLEHY/icinga2-2.4.1/lib/base/tlsstream.cpp:330
#7  0x00007ffff47849e4 in icinga::JsonRpcConnection::Disconnect (this=this@entry=0x7fff9c02b010) at /build/icinga2-upLEHY/icinga2-2.4.1/lib/remote/jsonrpcconnection.cpp:116
#8  0x00007ffff4788952 in icinga::JsonRpcConnection::DataAvailableHandler (this=0x7fff9c02b010) at /build/icinga2-upLEHY/icinga2-2.4.1/lib/remote/jsonrpcconnection.cpp:204
#9  0x00007ffff72baf2c in operator() (a0=..., this=) at /usr/include/boost/function/function_template.hpp:767
#10 m_invoke&)>, 0u, const boost::intrusive_ptr&> (args=..., func=..., this=)

at /usr/include/boost/signals2/detail/variadic_slot_invoker.hpp:92

#11 operator()&)>, const boost::intrusive_ptr&, 1ul> (args=..., func=..., this=)

at /usr/include/boost/signals2/detail/variadic_slot_invoker.hpp:81

#12 m_invoke >, boost::signals2::slot&), boost::function&)> >, boost::signals2::mutex> > > (connectionBody=..., this=)

at /usr/include/boost/signals2/detail/variadic_slot_invoker.hpp:117

#13 operator() >, boost::signals2::slot&), boost::function&)> >, boost::signals2::mutex> > > (connectionBody=..., this=)

at /usr/include/boost/signals2/detail/variadic_slot_invoker.hpp:110

#14 dereference (this=0x7fffe09d11c0) at /usr/include/boost/signals2/detail/slot_call_iterator.hpp:82
#15 dereference&>, std::_List_iterator >, boost::signals2::slot&), boost::function&)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body >, boost::signals2::slot&), boost::function&)> >, boost::signals2::mutex> > > (f=...) at /usr/include/boost/iterator/iterator_facade.hpp:514
#16 operator* (this=0x7fffe09d11c0) at /usr/include/boost/iterator/iterator_facade.hpp:639
#17 operator()&>, std::_List_iterator >, boost::signals2::slot&), boost::function&)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body >, boost::signals2::slot&), boost::function&)> >, boost::signals2::mutex> > > (last=..., first=..., this=) at /usr/include/boost/signals2/optional_last_value.hpp:55
#18 operator(), boost::signals2::detail::slot_call_iterator_t&>, std::_List_iterator >, boost::signals2::slot&), boost::function&)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body >, boost::signals2::slot&), boost::function&)> >, boost::signals2::mutex> > > (last=..., first=..., combiner=..., this=)

at /usr/include/boost/signals2/detail/result_type_wrapper.hpp:64

#19 boost::signals2::detail::signal_impl const&), boost::signals2::optional_last_value, int, std::less, boost::function const&)>, boost::function const&)>, boost::signals2::mutex>::operator()(boost::intrusive_ptr const&) (this=, args#0=...) at /usr/include/boost/signals2/detail/signal_template.hpp:246
#20 0x00007ffff7269620 in operator() (args#0=..., this=0x7fff9c0440a8) at /usr/include/boost/signals2/detail/signal_template.hpp:695
#21 icinga::Stream::SignalDataAvailable (this=this@entry=0x7fff9c044070) at /build/icinga2-upLEHY/icinga2-2.4.1/lib/base/stream.cpp:55
#22 0x00007ffff7269b08 in icinga::TlsStream::OnEvent (this=0x7fff9c044070, revents=) at /build/icinga2-upLEHY/icinga2-2.4.1/lib/base/tlsstream.cpp:241
#23 0x00007ffff72666aa in icinga::SocketEvents::ThreadProc () at /build/icinga2-upLEHY/icinga2-2.4.1/lib/base/socketevents.cpp:147
#24 0x00007ffff7bcfa4a in ?? () from /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.54.0
#25 0x00007ffff753c182 in start_thread (arg=0x7fffe09d2700) at pthread_create.c:312
#26 0x00007ffff63a947d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-01-27 13:19:58 +00:00

  • Relates set to 10931

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-01-29 09:27:01 +00:00

  • Category set to Cluster
  • Status changed from New to Assigned
  • Assigned to set to mfriedrich

@icinga-migration
Copy link
Author

Updated by bsheqa on 2016-02-09 07:29:45 +00:00

I could reproduce the problem with icinga running with gdb.
Here's the output of bt:

[2016-02-08 16:31:42 +0100] information/ConfigItem: Triggering Start signal for config items
[2016-02-08 16:31:42 +0100] information/ConfigItem: Activated all objects.
[Thread 0x7fff2979a700 (LWP 12433) exited]
icinga2: /usr/include/boost/smart_ptr/intrusive_ptr.hpp:162: T* boost::intrusive_ptr::operator->() const [with T = icinga::ConfigObject]: Assertion `px != 0' failed.

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7fff8b7fe700 (LWP 6033)]
0x00007ffff62e5cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56      ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.



(gdb) bt

#0  0x00007ffff62e5cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007ffff62e90d8 in __GI_abort () at abort.c:89
#2  0x00007ffff62deb86 in __assert_fail_base (fmt=0x7ffff642f830 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x7ffff47e22b8 "px != 0",

file=file@entry=0x7ffff47dfc30 "/usr/include/boost/smart_ptr/intrusive_ptr.hpp", line=line@entry=162,
function=function@entry=0x7ffff47f9280 <boost::intrusive_ptricinga::ConfigObject::operator~~() const::PRETTY_FUNCTION> "T* boost::intrusive_ptr::operator~~>() const [with T = icinga::ConfigObject]") at assert.c:92

#3  0x00007ffff62dec32 in __GI___assert_fail (assertion=0x7ffff47e22b8 "px != 0", file=0x7ffff47dfc30 "/usr/include/boost/smart_ptr/intrusive_ptr.hpp", line=162,

function=0x7ffff47f9280 <boost::intrusive_ptricinga::ConfigObject::operator~~() const::PRETTY_FUNCTION> "T* boost::intrusive_ptr::operator~~>() const [with T = icinga::ConfigObject]") at assert.c:101

#4  0x00007ffff4734cb3 in boost::intrusive_ptr::operator-> (this=0x7fff8b7fbb30) at /usr/include/boost/smart_ptr/intrusive_ptr.hpp:162
#5  0x00007ffff476f385 in operator-> (this=) at /usr/include/boost/smart_ptr/intrusive_ptr.hpp:97
#6  icinga::ApiListener::ConfigUpdateObjectAPIHandler (origin=..., params=...) at /build/icinga2-upLEHY/icinga2-2.4.1/lib/remote/apilistener-configsync.cpp:133
#7  0x00007ffff478dcdf in boost::detail::function::function_invoker2 const&, boost::intrusive_ptr const&), icinga::Value, boost::intrusive_ptr const&, boost::intrusive_ptr const&>::invoke (function_ptr=..., a0=..., a1=...)

at /usr/include/boost/function/function_template.hpp:95

#8  0x00007ffff473e68d in operator() (a1=..., a0=..., this=) at /usr/include/boost/function/function_template.hpp:767
#9  icinga::ApiFunction::Invoke (this=, origin=..., arguments=...) at /build/icinga2-upLEHY/icinga2-2.4.1/lib/remote/apifunction.cpp:31
#10 0x00007ffff476beb4 in icinga::JsonRpcConnection::ProcessMessage (this=this@entry=0x7fff5c011130) at /build/icinga2-upLEHY/icinga2-2.4.1/lib/remote/jsonrpcconnection.cpp:173
#11 0x00007ffff4788848 in icinga::JsonRpcConnection::DataAvailableHandler (this=0x7fff5c011130) at /build/icinga2-upLEHY/icinga2-2.4.1/lib/remote/jsonrpcconnection.cpp:197
#12 0x00007ffff72baf2c in operator() (a0=..., this=) at /usr/include/boost/function/function_template.hpp:767
#13 m_invoke&)>, 0u, const boost::intrusive_ptr&> (args=..., func=..., this=)

at /usr/include/boost/signals2/detail/variadic_slot_invoker.hpp:92

#14 operator()&)>, const boost::intrusive_ptr&, 1ul> (args=..., func=..., this=)

at /usr/include/boost/signals2/detail/variadic_slot_invoker.hpp:81

#15 m_invoke >, boost::signals2::slot&), boost::function&)> >, boost::signals2::mutex> > > (connectionBody=..., this=)

at /usr/include/boost/signals2/detail/variadic_slot_invoker.hpp:117

#16 operator() >, boost::signals2::slot&), boost::function&)> >, boost::signals2::mutex> > > (connectionBody=..., this=)

at /usr/include/boost/signals2/detail/variadic_slot_invoker.hpp:110

#17 dereference (this=0x7fff8b7fd1c0) at /usr/include/boost/signals2/detail/slot_call_iterator.hpp:82
#18 dereference&>, std::_List_iterator >, boost::signals2::slot&), boost::function&)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body >, boost::signals2::slot&), boost::function&)> >, boost::signals2::mutex> > > (f=...) at /usr/include/boost/iterator/iterator_facade.hpp:514
#19 operator* (this=0x7fff8b7fd1c0) at /usr/include/boost/iterator/iterator_facade.hpp:639
#20 operator()&>, std::_List_iterator >, boost::signals2::slot&), boost::function&)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body >, boost::signals2::slot&), boost::function&)> >, boost::signals2::mutex> > > (last=..., first=..., this=) at /usr/include/boost/signals2/optional_last_value.hpp:55
#21 operator(), boost::signals2::detail::slot_call_iterator_t&>, std::_List_iterator >, boost::signals2::slot&), boost::function&)> >, boost::signals2::mutex> > >, boost::signals2::detail::connection_body >, boost::signals2::slot&), boost::function&)> >, boost::signals2::mutex> > > (last=..., first=..., combiner=..., this=)

at /usr/include/boost/signals2/detail/result_type_wrapper.hpp:64
--Type to continue, or q to quit--

#22 boost::signals2::detail::signal_impl const&), boost::signals2::optional_last_value, int, std::less, boost::function const&)>, boost::function const&)>, boost::signals2::mutex>::operator()(boost::intrusive_ptr const&) (this=, args#0=...) at /usr/include/boost/signals2/detail/signal_template.hpp:246
#23 0x00007ffff7269620 in operator() (args#0=..., this=0x7fff5c00edd8) at /usr/include/boost/signals2/detail/signal_template.hpp:695
#24 icinga::Stream::SignalDataAvailable (this=this@entry=0x7fff5c00eda0) at /build/icinga2-upLEHY/icinga2-2.4.1/lib/base/stream.cpp:55
#25 0x00007ffff7269b48 in icinga::TlsStream::OnEvent (this=0x7fff5c00eda0, revents=) at /build/icinga2-upLEHY/icinga2-2.4.1/lib/base/tlsstream.cpp:197
#26 0x00007ffff72666aa in icinga::SocketEvents::ThreadProc () at /build/icinga2-upLEHY/icinga2-2.4.1/lib/base/socketevents.cpp:147
#27 0x00007ffff7bcfa4a in ?? () from /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.54.0
#28 0x00007ffff753c182 in start_thread (arg=0x7fff8b7fe700) at pthread_create.c:312
#29 0x00007ffff63a947d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

@icinga-migration
Copy link
Author

Updated by gbeutner on 2016-02-09 08:13:33 +00:00

  • Duplicates set to 10783

@icinga-migration
Copy link
Author

Updated by gbeutner on 2016-02-09 08:14:15 +00:00

  • Assigned to changed from mfriedrich to gbeutner
  • Target Version set to 2.4.2

@icinga-migration
Copy link
Author

Updated by Anonymous on 2016-02-09 08:14:33 +00:00

  • Status changed from Assigned to Resolved
  • Done % changed from 0 to 100

Applied in changeset 14d805e.

@icinga-migration
Copy link
Author

Updated by gbeutner on 2016-02-23 09:58:43 +00:00

  • Backport? changed from Not yet backported to Already backported

@icinga-migration
Copy link
Author

Updated by gbeutner on 2016-02-23 10:54:47 +00:00

  • Subject changed from Icinga2 Crash to Segfault in ApiListener::ConfigUpdateObjectAPIHandler

@icinga-migration icinga-migration added bug Something isn't working area/distributed Distributed monitoring (master, satellites, clients) labels Jan 17, 2017
@icinga-migration icinga-migration added this to the 2.4.2 milestone Jan 17, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/distributed Distributed monitoring (master, satellites, clients) bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant