Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[dev.icinga.com #11541] Objects created in a global zone are not synced to child endpoints #4098

Closed
icinga-migration opened this issue Apr 7, 2016 · 20 comments
Closed

[dev.icinga.com #11541] Objects created in a global zone are not synced to child endpoints #4098

icinga-migration opened this issue Apr 7, 2016 · 20 comments
Labels
area/api REST API blocker Blocks a release or needs immediate attention bug Something isn't working
Milestone
2.6.0

Comments

@icinga-migration
Copy link

This issue has been migrated from Redmine: https://dev.icinga.com/issues/11541

Created by Christian_vlc on 2016-04-07 12:47:01 +00:00

Assignee: mfriedrich
Status: Resolved (closed on 2016-11-14 13:45:03 +00:00)
Target Version: 2.6.0
Last Update: 2016-11-18 11:00:15 +00:00 (in Redmine)

Icinga Version: r2.4.3-1 2.4.4-1
Backport?: Not yet backported
Include in Changelog: 1

Hi,

Until now I configure every time the hostgroups in the zone "global-templates", and then they are sync to every other Zone (like the templates and contacts). But now I have one problem if I tried to build the hostgroup via API. All templates and contacts are sync but the hostgroups are sync only if are via Config File configured (package: _etc).

Is that one Bug? one limitation? or I'm doing something false?

my zones.conf (in both servers):

object Zone "master" {
endpoints = [ "icinga-dev-01.ipx" ]
}

object Zone "development" {
endpoints = [ "icinga-dev-03.ipx" ]
parent = "master"
}

object Zone "global-templates" {
  global = true
}

my api conf (in both servers):

object ApiListener "api" {
  cert_path = SysconfDir + "/icinga2/pki/" + NodeName + ".crt"
  key_path = SysconfDir + "/icinga2/pki/" + NodeName + ".key"
  ca_path = SysconfDir + "/icinga2/pki/ca.crt"

  ticket_salt = TicketSalt
  accept_config = true
  accept_commands = true

Group configured via API:

curl --silent -k --cert ls40311.crt --key ls40311.key 'https://icinga-dev-01.ipx:5665/v1/objects/hostgroups/testXX3'  | python -m json.tool
{
    "results": [
        {
            "attrs": {
                "__name": "testXX3", 
                "action_url": "", 
                "active": true, 
                "display_name": "testXX3", 
                "groups": null, 
                "ha_mode": 0.0, 
                "name": "testXX3", 
                "notes": "", 
                "notes_url": "", 
                "original_attributes": {
                    "display_name": "testXX3", 
                    "zone": "global-templates"
                }, 
                "package": "_api", 
                "paused": false, 
                "templates": [
                    "testXX3"
                ], 
                "type": "HostGroup", 
                "vars": null, 
                "version": 1460032265.482408, 
                "zone": "global-templates"
            }, 
            "joins": {}, 
            "meta": {}, 
            "name": "testXX3", 
            "type": "HostGroup"
        }
    ]
}

curl --silent -k --cert ls40311.crt --key ls40311.key 'https://icinga-dev-03.ipx:5665/v1/objects/hostgroups/testXX3'  | python -m json.tool
{
    "error": 404.0, 
    "status": "No objects found."
}

Group configured via _etc:

 curl --silent -k --cert ls40311.crt --key ls40311.key 'https://icinga-dev-01.ipx:5665/v1/objects/hostgroups/development'  | python -m json.tool
{
    "results": [
        {
            "attrs": {
                "__name": "development", 
                "action_url": "", 
                "active": true, 
                "display_name": "development", 
                "groups": null, 
                "ha_mode": 0.0, 
                "name": "development", 
                "notes": "", 
                "notes_url": "", 
                "original_attributes": null, 
                "package": "_etc", 
                "paused": false, 
                "templates": [
                    "development"
                ], 
                "type": "HostGroup", 
                "vars": null, 
                "version": 0.0, 
                "zone": "global-templates"
            }, 
            "joins": {}, 
            "meta": {}, 
            "name": "development", 
            "type": "HostGroup"
        }
    ]
}

curl --silent -k --cert ls40311.crt --key ls40311.key 'https://icinga-dev-03.ipx:5665/v1/objects/hostgroups/development'  | python -m json.tool
{
    "results": [
        {
            "attrs": {
                "__name": "development", 
                "action_url": "", 
                "active": true, 
                "display_name": "development", 
                "groups": null, 
                "ha_mode": 0.0, 
                "name": "development", 
                "notes": "", 
                "notes_url": "", 
                "original_attributes": null, 
                "package": "_etc", 
                "paused": false, 
                "templates": [
                    "development"
                ], 
                "type": "HostGroup", 
                "vars": null, 
                "version": 0.0, 
                "zone": "global-templates"
            }, 
            "joins": {}, 
            "meta": {}, 
            "name": "development", 
            "type": "HostGroup"
        }
    ]
}

Icinga Version: r2.4.3-1 and 2.4.4-1 (I test it with these two version)

**** I use every time the API of the server icinga-dev-01 (master Zone) for configure the Objects

Many Thanks,
Christian

Changesets

2016-11-14 13:42:47 +00:00 by mfriedrich 40d68fc

Fix object sync for objects in a global zone

fixes #11541

2016-11-18 13:00:41 +00:00 by mfriedrich ca7f195

Remove debug log message

refs #11541

Relations:
@icinga-migration
Copy link
Author

Updated by Christian_vlc on 2016-04-07 12:50:36 +00:00

sorry, I did one mistake with the copy/paste:

Group configured via _etc:

curl --silent -k --cert ls40311.crt --key ls40311.key 'https://icinga-dev-01.ipx:5665/v1/objects/hostgroups/development'  | python -m json.tool
{
    "results": [
        {
            "attrs": {
                "__name": "development", 
                "action_url": "", 
                "active": true, 
                "display_name": "development", 
                "groups": null, 
                "ha_mode": 0.0, 
                "name": "development", 
                "notes": "", 
                "notes_url": "", 
                "original_attributes": null, 
                "package": "_etc", 
                "paused": false, 
                "templates": [
                    "development"
                ], 
                "type": "HostGroup", 
                "vars": null, 
                "version": 0.0, 
                "zone": "global-templates"
            }, 
            "joins": {}, 
            "meta": {}, 
            "name": "development", 
            "type": "HostGroup"
        }
    ]
}


curl --silent -k --cert ls40311.crt --key ls40311.key 'https://icinga-dev-03.ipx:5665/v1/objects/hostgroups/development'  | python -m json.tool
{
    "results": [
        {
            "attrs": {
                "__name": "development", 
                "action_url": "", 
                "active": true, 
                "display_name": "development", 
                "groups": null, 
                "ha_mode": 0.0, 
                "name": "development", 
                "notes": "", 
                "notes_url": "", 
                "original_attributes": null, 
                "package": "_etc", 
                "paused": false, 
                "templates": [
                    "development"
                ], 
                "type": "HostGroup", 
                "vars": null, 
                "version": 0.0, 
                "zone": "global-templates"
            }, 
            "joins": {}, 
            "meta": {}, 
            "name": "development", 
            "type": "HostGroup"
        }
    ]
}

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-04-07 13:50:57 +00:00

  • Status changed from New to Assigned
  • Assigned to set to mfriedrich

CanAccessZone() needs to return true for objects which are in a global zone.

@icinga-migration
Copy link
Author

Updated by Christian_vlc on 2016-04-07 14:03:47 +00:00

Ok, if you need something... debug.log or some other thing, please tell me :-)

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-05-21 13:58:46 +00:00

  • Description updated

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-05-21 13:59:33 +00:00

Please use the pre-Tags for configuration/commands/logs next time.

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-05-21 14:23:35 +00:00

Steps to reproduce:

curl -X "PUT" "https://localhost:7000/v1/objects/hostgroups/global-group" \
        -H "Content-Type: text/plain" \
        -H "Authorization: Basic cm9vdDoqKioqKiBIaWRkZW4gY3JlZGVudGlhbHMgKioqKio=" \
        -H "Accept: application/json" \
        -d $'{
  "attrs": {
    "zone": "global-templates"
  }
}'

mbmif /usr/local/tests/icinga2/master-slave (master) # tree icinga2*/lib/icinga2/api/packages/_api/*/
icinga2a/lib/icinga2/api/packages/_api/mbmif.int.netways.de-1442332428-0/
├── conf.d
│   ├── comments
│   ├── downtimes
│   ├── hostgroups
│   │   └── global-group.conf
│   └── hosts
│       ├── google.com10.conf
│       ├── google.com11.conf
│       ├── google.com2.conf
│       ├── google.com3.conf
│       ├── google.com6.conf
│       ├── google.com7.conf
│       ├── google.dns.conf
│       ├── google.zone.conf
│       └── google1.conf
├── include.conf
└── zones.d
    └── master
        └── hosts
            └── api-sync-host.conf
icinga2b/lib/icinga2/api/packages/_api/mbmif.local-1463837930-0/
├── conf.d
│   └── hosts
│       ├── google.com11.conf
│       ├── google.com2.conf
│       └── google.com3.conf
├── include.conf
└── zones.d
icinga2c/lib/icinga2/api/packages/_api/mbmif.int.netways.de-1442488228-0/
└── conf.d
    ├── comments
    ├── downtimes
    │   ├── satellite-host-02!mbmif.int.netways.de-1459245818-0.conf
    │   └── satellite-host-02!mbmif.int.netways.de-1459246170-2.conf
    └── hosts
        ├── google.com11.conf
        ├── google.com2.conf
        └── google.com3.conf

15 directories, 21 files

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-05-21 14:23:54 +00:00

  • Relates set to 11684

@icinga-migration
Copy link
Author

Updated by Christian_vlc on 2016-05-24 06:35:00 +00:00

Hi Michael, do you have some idea about when can be it fixed? is there a targeted fix for it? Sorry, I don't want pressure you.. I only want know if I can wait or if I should try some workaround for our register process via API :)

@icinga-migration
Copy link
Author

Updated by Christian_vlc on 2016-06-23 13:19:22 +00:00

Hi!

are some update? maybe one target version or temporal solution? We cant use the API for create new Hosts/Hostsgroups in one cluster environment (with differents Zones)

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-06-23 13:38:30 +00:00

  • Priority changed from Normal to High

The required changes are larger than expected. We've put a higher priority onto it but still dealing with other issues at the time of writing.

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-06-23 13:38:41 +00:00

  • Relates deleted 11684

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-06-23 13:38:54 +00:00

  • Parent Id set to 11415

@icinga-migration
Copy link
Author

Updated by Christian_vlc on 2016-06-24 08:25:48 +00:00

ok, many thanks for the Info and for the higher priority Michael :)

@icinga-migration
Copy link
Author

Updated by Christian_vlc on 2016-09-22 11:54:35 +00:00

Hi Michael,
how looks the ticket? are maybe some target version?

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-09-28 13:43:35 +00:00

  • Target Version set to 2.6.0
  • Parent Id deleted 11415

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-11-10 19:05:18 +00:00

  • Relates set to 11684

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-11-11 10:24:17 +00:00

Resync works now with fixes applied in #11684. Initial synchronisation fails due to messages not relayed to any node, if the target_zone is global.

[2016-11-11 11:22:18 +0100] information/HttpServerConnection: Request: PUT /v1/objects/users/global3 (from [127.0.0.1]:51765, user: root)
[2016-11-11 11:22:18 +0100] debug/HttpUtility: Request body: '{ "attrs": { "zone": "global-templates" } }'
[2016-11-11 11:22:18 +0100] notice/ConfigCompiler: Compiling config file: icinga2a/lib/icinga2/api/packages/_api/mbmif.int.netways.de-1442332428-0/conf.d/users/global3.conf
[2016-11-11 11:22:18 +0100] information/ConfigItem: Committing config item(s).
[2016-11-11 11:22:18 +0100] notice/WorkQueue: Spawning WorkQueue threads for ''
[2016-11-11 11:22:18 +0100] debug/ConfigItem: Commit called for ConfigItem Type=User, Name=global3
[2016-11-11 11:22:18 +0100] information/ConfigItem: Instantiated 1 User.
[2016-11-11 11:22:18 +0100] information/ConfigItem: Triggering Start signal for config items
[2016-11-11 11:22:18 +0100] debug/ConfigItem: Activating object 'global3' of type 'User'
[2016-11-11 11:22:18 +0100] debug/ApiListener: Sent update for object 'global3': {"config":"object User \"global3\" {\n\tversion = 1478859738.854025\n\tzone = \"global-templates\"\n}\n","modified_attributes":{},"name":"global3","original_attributes":[],"type":"User","version":1478859738.8540248871}
[2016-11-11 11:22:18 +0100] notice/ApiListener: Relaying 'config::UpdateObject' message
[2016-11-11 11:22:18 +0100] information/ConfigItem: Activated all objects.
[2016-11-11 11:22:18 +0100] critical/ApiListener: Not relaying message 'config::UpdateObject'. Not in the same/parent/child zone.

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-11-11 14:04:51 +00:00

This affects message routing for global target zones. Currently we do not "broadcast" messages to a global zone which literally means to any node. Limitations apply to all parent and child zone members, they take then care of re-routing the messages.

That's not cached at all and might cause trouble with fetching all immediate parent and child zone members on sending just one cluster message. A patch is located in fix/api-cluster-global-zone-11541 but I am not sure if I really want that being released.

The fairly small other required patch is to open CanAccessObject() for objects in a global zone, which basically is "always true"). The difficult parts remain inside apilistener.cpp

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-11-14 13:45:03 +00:00

  • Status changed from Assigned to Resolved
  • Done % changed from 0 to 100

Applied in changeset 40d68fc.

@icinga-migration
Copy link
Author

Updated by mfriedrich on 2016-11-18 11:00:15 +00:00

  • Subject changed from hostgroups that are configured via API in global-templates doesnt sync to other Zones to Objects created in a global zone are not synced to child endpoints

@icinga-migration icinga-migration added blocker Blocks a release or needs immediate attention bug Something isn't working area/api REST API labels Jan 17, 2017
@icinga-migration icinga-migration added this to the 2.6.0 milestone Jan 17, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/api REST API blocker Blocks a release or needs immediate attention bug Something isn't working
Projects
None yet
Milestone
2.6.0
Development

No branches or pull requests
1 participant
@icinga-migration