yaml/yq parser round-trip behavior: unmarshaling of integer keys in dicts

[robbat2]

TL;DR: YQ tries to follow the yaml spec and quote integer keys in certain places of a YAML file, esp. dict keys. This causes a parsing error.

• The existing snmp.yml & snmp.local.yml model does not provide a good way to override a single attribute in a module.
• It can't be copied and tweaked in snmp.local.yml, as causes an error about duplicate module.
• There's no clean patch mechanism either.

As a workaround, I was going to ship a transformation using YQ, that converts the stock snmp.yml, with a filter, to produce the site-specific snmp.yml. However, I found that the YAML generated by YQ, with either the -y or -Y options, differs slightly in quoting of enum_values. And that causes snmp_exporter to fail startup.

--- snmp.yml.example    2023-12-26 22:33:06.108558846 +0000
+++ snmp.yml    2024-02-14 20:57:04.721555438 +0000
@@ -78,9 +77,9 @@
       - labels: []
         labelname: upsOutletGroupStatusIndex
       enum_values:
-        1: upsOutletGroupStatusOn
-        2: upsOutletGroupStatusOff
-        3: upsOutletGroupStatusUnknown
+       '1': upsOutletGroupStatusOn
+       '2': upsOutletGroupStatusOff
+       '3': upsOutletGroupStatusUnknown
     - name: upsOutletGroupStatusCommandPending
       oid: 1.3.6.1.4.1.318.1.1.1.12.1.2.1.4
       type: gauge

(same change for every enum_values)

Host operating system: output of uname -a

Linux meadowlark 6.1.38-gentoo-dist #1 SMP PREEMPT_DYNAMIC Thu Jul 6 11:45:23 -00 2023 x86_64 QEMU Virtual CPU version 2.5+ GenuineIntel GNU/Linux

snmp_exporter version: output of snmp_exporter -version

0.24.1

What did you do that produced an error?

yq . <snmp.yaml.example >snmp.yaml
snmp_exporter --config.file=snmp.yml

What did you expect to see?

snmp_exporter should have started

What did you see instead?

ts=2024-02-14T20:48:17.765Z caller=main.go:202 level=error msg="Error parsing config file" err="yaml: unmarshal errors:\n  line 80: cannot unmarshal !!str `1` into int\n  line 81: cannot unmarshal !!str `2` into int\n  line 82: cannot unmarshal !!str `3` into int\n  line 100: cannot unmarshal !!str `1` into int\n  line 101: cannot unmarshal !!str `2` into int\n  line 118: cannot unmarshal !!str `1` into int\n  line 119: cannot unmarshal !!str `2` into int\n  line 193: cannot unmarshal !!str `1` into int\n  line 194: cannot unmarshal !!str `2` into int\n  line 204: cannot unmarshal !!str `1` into int\n  line 205: cannot unmarshal !!str `2` into int\n  line 215: cannot unmarshal !!str `1` into int\n  line 216: cannot unmarshal !!str `2` into int\n  line 227: cannot unmarshal !!str `1` into int\n  line 228: cannot unmarshal !!str `2` into int\n  line 239: cannot unmarshal !!str `1` into int\n  line 240: cannot unmarshal !!str `2` into int\n  line 250: cannot unmarshal !!str `1` into int\n ...

[SuperQ]

The existing snmp.yml & snmp.local.yml model does not provide a good way to override a single attribute in a module.

I'm not sure what you mean by this, you can always use the generator to overwrite your config files however you want. The exporter has no concept of snmp.local.yml.

I'm not sure how flexible the yaml unmarshal behavior is. It may be possible to change this to parse the strings.

[SuperQ]

Reading over the YAML spec, keys can be any kind of scaler, which doesn't seem to indicate that they can't be integers. I don't see what is done right now is a violation of any YAML spec.

I think yq is doing something wrong here.