[dev.icinga.com #10554] Non-UTF8 characters from plugins causes IDO to fail

[icinga-migration]

This issue has been migrated from Redmine: https://dev.icinga.com/issues/10554

Created by dgoetz on 2015-11-06 12:35:23 +00:00

Assignee: gbeutner
Status: Resolved (closed on 2015-12-10 11:30:04 +00:00)
Target Version: 2.4.2
Last Update: 2016-02-23 09:58:17 +00:00 (in Redmine)

Icinga Version: 2.3.11
Backport?: Already backported
Include in Changelog: 1

---

A plugin giving output with non-UTF8 characters causes IDO to fail. Because we can not make sure every plugin handles this correctly. Can we get Icinga 2 to ignore such plugin output?

An example:

[2015-11-06 13:08:17 +0100] critical/IdoPgsqlConnection: Error "ERROR: 
invalid byte sequence for encoding "UTF8": 0xf6 0x73 0x74 0x65
" when executing query "UPDATE icinga_servicestatus SET 
acknowledgement_type = E'0',  active_checks_enabled = E'1',  check_command 
= E'nrpe',  check_source = E'su01k027.xxx.XXX.de', 
check_timeperiod_object_id = 79306,  check_type = E'0', 
current_check_attempt = E'1',  current_notification_number = E'2', 
current_state = E'1',  endpoint_object_id = 79031,  event_handler = E'', 
event_handler_enabled = E'1',  execution_time = E'0.62169599533081055', 
flap_detection_enabled = E'0',  has_been_checked = E'1',  instance_id = 1, 
 is_flapping = E'0',  is_reachable = E'1',  last_check = 
TO_TIMESTAMP(1446790307),  last_hard_state = E'1',  last_hard_state_change 
= TO_TIMESTAMP(1446798595),  last_notification = TO_TIMESTAMP(1446801203), 
 last_state_change = TO_TIMESTAMP(1446798595),  last_time_ok = 
TO_TIMESTAMP(1443419904),  last_time_warning = TO_TIMESTAMP(1446798595), 
latency = E'0',  long_output = E'',  max_check_attempts = E'1', 
modified_service_attributes = E'0',  next_check = 
TO_TIMESTAMP(1446876284),  next_notification = TO_TIMESTAMP(1446815603), 
normal_check_interval = E'1440',  notifications_enabled = E'1',  output = 
E'WARNING - (261 warnings) - 11/06/2015 04:45:43 ANS1228E Sending of 
object 
''/home/aea11/dat/domino/html/sht/www/images/europakarte/karte▒ster.JPG'' 
failed ...',  passive_checks_enabled = E'1',  percent_state_change = 
E'99',  perfdata = E'dsmerror_log_lines=10348 dsmerror_log_warnings=261 
dsmerror_log_criticals=0 dsmerror_log_unknowns=0', 
problem_has_been_acknowledged = E'0',  process_performance_data = E'1', 
retry_check_interval = E'1',  scheduled_downtime_depth = E'0', 
service_object_id = 94836,  should_be_scheduled = E'1',  state_type = 
E'1',  status_update_time = TO_TIMESTAMP(1446811697) WHERE 
service_object_id = 94836"
Context:
        (0) Reconnecting to PostgreSQL IDO database 'ido-pgsql'

Attachments

• i2_10554_pgsql_backend.png mfriedrich - 2015-12-10 11:38:00 +00:00
• i2_10554_fixed.png mfriedrich - 2015-12-10 11:38:07 +00:00

Changesets

2015-12-10 11:26:19 +00:00 by (unknown) fba0b49

Make sure that all strings in the IDO database are UTF8-encoded

fixes #10554

2016-02-23 08:08:32 +00:00 by (unknown) 2e93082

Make sure that all strings in the IDO database are UTF8-encoded

fixes #10554

[icinga-migration]

Updated by gbeutner on 2015-11-18 08:49:35 +00:00

• Category set to DB IDO

[icinga-migration]

Updated by gbeutner on 2015-11-18 08:59:36 +00:00

The underlying problem is that there's no way to determine which encoding things like plugin output and config object names are in.

One idea would be to set the client encoding to latin1:

set client_encoding to 'latin1'

[icinga-migration]

Updated by gbeutner on 2015-11-18 19:19:06 +00:00

• Status changed from New to Feedback
• Assigned to set to mfriedrich

[icinga-migration]

Updated by mfrosch on 2015-11-23 09:04:30 +00:00

• Description updated

[icinga-migration]

Updated by mfrosch on 2015-11-23 15:56:49 +00:00

Best way seems to be setting connection encoding to 'latin1', since the IDO is modelled for latin.

Though it needs testing if we eradicate these problems with that change.

Icinga 1 would have similar problems with incorrect chars, but will write it to database, due to latin1.

[icinga-migration]

Updated by mfriedrich on 2015-11-23 16:15:19 +00:00

• Status changed from Feedback to New
• Assigned to deleted mfriedrich
• Priority changed from Normal to High
• Target Version set to Backlog

[icinga-migration]

Updated by mfriedrich on 2015-11-24 13:04:52 +00:00

Quick workaround: Fix the plugin output and strip non-utf8 characters before passing the output back to Icinga 2.

[icinga-migration]

Updated by mfriedrich on 2015-12-10 11:27:02 +00:00

• Status changed from New to Assigned
• Assigned to set to gbeutner
• Target Version changed from Backlog to 2.4.2

[icinga-migration]

Updated by Anonymous on 2015-12-10 11:30:04 +00:00

• Status changed from Assigned to Resolved
• Done % changed from 0 to 100

Applied in changeset fba0b49.

[icinga-migration]

Updated by mfriedrich on 2015-12-10 11:39:43 +00:00

• File added i2_10554_pgsql_backend.png
• File added i2_10554_fixed.png

Fixed

[icinga-migration]

Updated by gbeutner on 2016-02-23 09:58:17 +00:00

• Backport? changed from Not yet backported to Already backported