Description
Describe the bug:
When using the Concat Filter, I think the regexp value for the field multiline_end_regexp
is not properly formatted into the generated Fluentd config :
My Flow configuration:
apiVersion: v1
- apiVersion: logging.banzaicloud.io/v1beta1
kind: Flow
metadata:
name: my-flow
namespace: xxx
spec:
filters:
- concat:
key: log
multiline_end_regexp: /\n$/
separator: ""
use_first_timestamp: true
- grep:
regexp:
- key: log
pattern: /^\{.*\}/
- parser:
key_name: log
parse:
type: json
remove_key_name_field: true
reserve_data: true
localOutputRefs:
- my-output
match:
- select:
container_names:
- xxx
The resulting FluentdConfig (taken from the logs of the pod rancher-logging-root-fluentd-configcheck
) has the value "/\\n$/"
for the multiline_end_regexp
field :
<label @2b0669f6af9ddff877bae45460fe9d59>
<filter **>
@type concat
@id flow:xxx:my-flow:0
key "log"
multiline_end_regexp "/\\n$/"
use_first_timestamp true
</filter>
<filter **>
@type grep
@id flow:xxx:my-flow:1
<regexp>
key "log"
pattern /^\{.*\}/
</regexp>
</filter>
<filter **>
@type parser
@id flow:xxx:my-flow:2
key_name "log"
remove_key_name_field true
reserve_data true
<parse>
@type "json"
</parse>
</filter>
<match **>
@type elasticsearch_data_stream
Expected behaviour:
I expect to obtain the value /\n$/
in the Fluentd config :
<filter **>
@type concat
@id flow:xxx:my-flow:0
key "log"
multiline_end_regexp /\n$/
use_first_timestamp true
</filter>
Steps to reproduce the bug:
- Create a flow with the above Flow configuration
- Check the generated Fluentd configuration
Additional context:
I'm dealing with JSON logs which have a size greater than 16 Kib an Docker splits them in multiples logs. I was trying to solve this problem by adding this Fluentd configuration but when porting it to my Flow
resource, the regexp for the field multiline_end_regexp
is not formatted properly in the resulting Fluentd config.
Environment details:
- Kubernetes version: v1.25.14
- Cloud-provider/provisioner: RKE1
- logging-operator version: 3.17.10
- Install method (e.g. helm or static manifests): Rancher Logging plugin (chart is
rancher-logging:102.0.3+up3.17.10
) - Logs from the misbehaving component (and any other relevant logs): /
- Resource definition (possibly in YAML format) that caused the issue, without sensitive data: see above
/kind bug