Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dev.icinga.com #10891] is_active in IDO is only re-enabled on "every second" restart #3798

Closed
icinga-migration opened this issue Dec 18, 2015 · 14 comments
Labels
area/db-ido Database output bug Something isn't working
Milestone

Comments

@icinga-migration
Copy link

This issue has been migrated from Redmine: https://dev.icinga.com/issues/10891

Created by tgelf on 2015-12-18 14:38:12 +00:00

Assignee: gbeutner
Status: Resolved (closed on 2016-01-11 13:25:02 +00:00)
Target Version: 2.4.2
Last Update: 2016-03-16 08:58:07 +00:00 (in Redmine)

Icinga Version: 2.4.1
Backport?: Already backported
Include in Changelog: 1

Looks like a new race condition...

Cheers,
Thomas

Changesets

2016-01-11 13:23:16 +00:00 by (unknown) 1bf2701

Fix incorrect is_active queries

fixes #10891

2016-02-23 08:18:24 +00:00 by (unknown) 11dfcd7

Fix incorrect is_active queries

fixes #10891

Relations:

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2015-12-18 17:17:55 +00:00

  • Category set to DB IDO
  • Target Version set to 2.4.2

@icinga-migration
Copy link
Author

Updated by elippmann on 2015-12-21 11:51:00 +00:00

I can confirm this.

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2015-12-22 10:58:36 +00:00

  • Priority changed from High to Urgent

We'll look into that after our xmas holidays (11.1.2016).

@icinga-migration
Copy link
Author

Updated by mthode on 2015-12-24 04:32:26 +00:00

Do you have more info on this? I think I'm hitting it, but don't know.

@icinga-migration
Copy link
Author

Updated by tgelf on 2015-12-24 08:53:18 +00:00

Unfortunately I guess because of Christmas holidays there will not happen much on this issue within the next weeks. I tried to work around this problem with a cronjob running once a minute doing as follows:

UPDATE icinga_objects o join icinga_hoststatus s ON s.host_object_id =
o.object_id set o.is_active = 1 where o.objecttype_id = 1 and
o.is_active = 0 AND s.last_check > NOW() - INTERVAL 1 MINUTE;
UPDATE icinga_objects o join icinga_servicestatus s ON
s.service_object_id = o.object_id set o.is_active = 1 where
o.objecttype_id = 1 and o.is_active = 0 AND s.last_check > NOW() -
INTERVAL 1 MINUTE;

Only drawback: in theory it could happen that this re-enables a deleted object in IDO, let me know when this happens to you - I could try to add a few more checks to the query.

Cheers,
Thomas

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2015-12-24 09:55:03 +00:00

This problem only exists in the current git master and does not affect stable releases.

As said, we are going to look into it after the holiday season enjoying our offline time for a while.

Happy holidays!

Michael

@icinga-migration
Copy link
Author

Updated by tgelf on 2015-12-24 10:09:32 +00:00

@mthode: Be assured that Michi and Gunnar absolutely deserve their holidays :-) I'd personally still suggest to stay with the latest snapshot if the above queries help to mitigate your problem. They did a very good job in fixing a lot of severe issues in 2.4.x, so staying with 2.4.1 was not an option for me.

Cheers,
Thomas

@icinga-migration
Copy link
Author

Updated by gbeutner on 2016-01-11 13:24:57 +00:00

  • Status changed from New to Assigned
  • Assigned to set to gbeutner

@icinga-migration
Copy link
Author

Updated by Anonymous on 2016-01-11 13:25:02 +00:00

  • Status changed from Assigned to Resolved
  • Done % changed from 0 to 100

Applied in changeset 1bf2701.

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-01-11 13:49:16 +00:00

Happens randomly, and returns a non-predictive number of active/inactive objects (http://localhost/icingaweb2/monitoring/list/services). The race condition happens inside the DbType::GetOrCreateByObjectName() not properly setting the object reference.

Verified working.

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-02-04 16:13:13 +00:00

  • Relates set to 11065

@icinga-migration
Copy link
Author

Updated by gbeutner on 2016-02-23 09:58:35 +00:00

  • Backport? changed from Not yet backported to Already backported

@icinga-migration
Copy link
Author

Updated by dokon on 2016-03-16 08:57:05 +00:00

Bug seems to be still there ... have a customer Setup where i get the following

----------~~------------------------------------------------------------------~~-----------+

object_id instance_id objecttype_id name1 name2 is_active

----------~~------------------------------------------------------------------~~-----------+

559 1 13 super.anonymous.com NULL 1
565 1 1 super.anonymous.com NULL 0

----------~~------------------------------------------------------------------~~-----------+

And the host is missing in the icingaweb2 host view , but viewable in icingaclassic... :-/

@icinga-migration
Copy link
Author

Updated by gbeutner on 2016-03-16 08:58:07 +00:00

Please submit a new ticket for this.

@icinga-migration icinga-migration added Urgent bug Something isn't working area/db-ido Database output labels Jan 17, 2017
@icinga-migration icinga-migration added this to the 2.4.2 milestone Jan 17, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/db-ido Database output bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant