[dev.icinga.com #12539] Thoughts about a new diagnostic tool

[icinga-migration]

This issue has been migrated from Redmine: https://dev.icinga.com/issues/12539

Created by sru on 2016-08-25 05:43:21 +00:00

Assignee: (none)
Status: New
Target Version: (none)
Last Update: 2016-09-04 13:02:07 +00:00 (in Redmine)

Backport?: Not yet backported
Include in Changelog: 1

---

current situation:

icinga2 object list

• easy to use
• can be filtered with textutils
• can be piped into a file that can be mailed to support / forum
• shows in which file an object has been declared / modified

• incomplete regarding api created objects

api query via curl

• is considered to show complete config object information
• can be filtered with textutils
• can be piped into a file that can be mailed to support / forum

• hard to use
• does not show in which file an object has been declared / modified
  ? does not show the content of templates (but doc states it is possible)

icinga studio as a frontend to api query

• is considered to show complete config object information
• easy to use

• can not be filtered with textutils
• can not be piped into a file that can be mailed to support / forum
• does not show in which file an object has been declared / modified
• does not show the content of templates

verdict:

• currently, we have no single tool to gather diagnostic information

• icinga2 object list does not inform the user that its information is valid but may be incomplete.

• api query from the commandline is too complex to be used occasionally and misses templates ? and physical locations

additional features:

• Benchmark function to answer the question: "how much checks per minute is my box able to handle ?"
  that being done for the following scenarios:

• Master with ido feature but not checking itself (perhaps with a switch to enable multible masters)

• Satellite standalone running synthetic checks - the later configureable to produce a delay in ms and / or a load per core in %cpu.

• Two satellites in a load sharing scenario, two see the impact of different hardware / decreasing system resources (cpu, ram, nic)

• list "status" api call output and system hardware

---

Relations:

• relates #12566

[icinga-migration]

Updated by gbeutner on 2016-08-27 17:33:58 +00:00

FYI, #12566 now provides location info for both objects and templates in the API.

As for template contents As far as Icinga is concerned templates are code - there are no attribute values until a particular template has been evaluated as part of another object definition. Let me give you an example, what would the attributes be for the following template?:

template Host "x" {
  check_interval = Math.random() * 300
}

Every time you evaluate this template you get a different attribute value - and this doesn't even begin to scratch the surface (think... if-then-else).

Benchmarks are somewhat out-of-scope for Icinga 2 and they usually don't represent real world scenarios. For example I've spent quite a bit of time on improving the IDO and config parser performance for 2.5.x using "real" customer configs (with their permission, of course) and have found bottlenecks I would've never found with a synthetic config.

Also, as for "icinga2 object list" I'd love to provide incremental updates (that is, for objects that were created at runtime). IIRC there's a ticket for that somewhere. Unfortunately this is somewhat non-trivial and would require a new file format for the icinga2.debug file (ideally, with indexes) which might be something we could do for 2.6.

Other than that I'm not sure what you're proposing here, i.e. what exactly is it you want for this feature request? :)

[icinga-migration]

Updated by gbeutner on 2016-08-27 17:34:13 +00:00

• Relates set to 12566

[icinga-migration]

Updated by sru on 2016-08-29 10:09:20 +00:00

Gunnar,

#12566 is solved: This will be of great help.

+As for template contents
Understood. But i am still after what lines of code are in the template. If we have the
location now, the tool can show this information from that file - as long it is run locally.

Benchmarks:
I agree that there are lots of things that have an impact on the performance.
I just wanted to bring in that there are forum questions to that and that we have nothing to answer than:

"Try it yourself. But normally your hardware will do."
Even harder that hits the HA / Load Balancing scenarios, in which it seems to be known nearly nothing about the impact of heavily different hardware resources.

What am i proposing here ?
Nothing.
Well, the "icinga object list" disclaimer "This tool currently does not show all (api related ?) information." would be nice.

I collect Forum Posts of

• Michael regarding a new diagnostic tool (as a retirement for icinga2 object list),

• Gunnar regarding the missing location info (meanwhile fixed, thanks again)

• As well as my experience regarding the work with the different support tools and what these lack from my point of view.

[icinga-migration]

Updated by mfriedrich on 2016-09-04 13:02:07 +00:00

homebrew provides a diagnostic tool which primarily checks for permission errors on directories ("brew doctor").
https://github.com/Homebrew/brew/blob/master/share/doc/homebrew/README.md

That would require changing the code in a way of registering required paths and permissions in a central location. Then such a cli tool could check for those, and possibly also fix them if told.

[dnsmichi]

I'll hand that to @lippserd for further planning on such a tool.

[gunnarbeutner]

There are no plans to support this.