system-config: add option forced_stacktrace_level #589
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
for v1.19 |
daipom
force-pushed
the
systemconfig-add-option-forced_stacktrace_level
branch
from
f8546e3 to
58221a1
Compare
fluent/fluentd#5008 Signed-off-by: Daijiro Fukuda <[email protected]>
daipom
force-pushed
the
systemconfig-add-option-forced_stacktrace_level
branch
from
58221a1 to
4f9177d
Compare
kenhys
requested changes
Jun 25, 2025
daipom
added a commit
to fluent/fluentd
that referenced
this pull request
Jun 25, 2025
**Which issue(s) this PR fixes**:
None.
**What this PR does / why we need it**:
**What this PR does**
Add a new system_config option: `forced_stacktrace_level`.
* Option: `system/log/forced_stacktrace_level`
* Type: `enum`
* `none`, `trace`, `debug`, `info`, `warn`, `error`, `fatal`
* Default: `none`
By default, the behavior of Fluentd does not change.
So, this does not affect existing users.
If you set this option and change the value, the log levels of
stacktraces are forced to that value.
Stacktraces that do not meet the log level are discarded in advance.
After the log level is changed, it is judged again for logging or
discarded finally.
For example, by default (`log_level info`), ...
* All `trace`/`debug` stacktraces are not logged regardless of this
option.
* If you set `forced_stacktrace_level debug`, then all stacktraces are
not logged.
This is because we should avoid logging logs that contradict
`log_level`, and to avoid unexpectedly logging logs that are initially
`trace`/`debug` level.
Setting example:
```xml
<system>
<log>
forced_stacktrace_level info
</log>
</system>
```
**why we need it**
To make it easier to exclude stacktraces from monitoring Fluentd's own
logs.
(You don't need this feature if using
[@FLUENT_LOG](https://docs.fluentd.org/deployment/logging#capture-fluentd-logs)
to transfer. For some reason, there are users who need to monitor
Fluentd's log files directly. This feature is for those uses.)
For example, there could be a system that raises an alert when an
error-level log is detected.
You may want to exclude some error-level logs from the detection.
However, if stacktraces are logged at error level, you need to exclude
those stacktraces as well.
With the current specification, that is troublesome.
Ideally, it would be preferable to exclude all stacktraces from the
detection.
It is sufficient to have access to stacktraces when investigating the
details.
An easy way to improve the current situation is this feature.
If you are detecting error logs as in the previous example, you can
easily exclude all stacktraces from the detection by forcing the
stacktrace level to `info` with this feature.
**Docs Changes**:
* fluent/fluentd-docs-gitbook#589
**Release Note**:
System configuration: Add `forced_stacktrace_level` to force the log
level of stacktraces.
**Sample**:
```xml
<system>
<log>
forced_stacktrace_level info
</log>
</system>
<source>
@type sample
@id test
tag test.fail
</source>
<match test.fail>
@type null
never_flush
</match>
```
Without `forced_stacktrace_level info`:
```
2025-06-23 18:16:45 +0900 [info]: #0 fluent/log.rb:371:info: fluentd worker is now running worker=0
2025-06-23 18:16:46 +0900 [warn]: #0 fluent/log.rb:392:warn: emit transaction failed: error_class=RuntimeError error="failed to flush" location="/home/daipom/work/fluentd/fluentd/lib/fluent/plugin/out_null.rb:54:in `process'" tag="test.fail"
2025-06-23 18:16:46 +0900 [warn]: #0 fluent/event_router.rb:114:emit_stream: /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/out_null.rb:54:in `process'
2025-06-23 18:16:46 +0900 [warn]: #0 fluent/event_router.rb:114:emit_stream: /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:865:in `emit_sync'
2025-06-23 18:16:46 +0900 [warn]: #0 fluent/event_router.rb:114:emit_stream: /home/daipom/work/fluentd/fluentd/lib/fluent/event_router.rb:115:in `emit_stream'
2025-06-23 18:16:46 +0900 [warn]: #0 fluent/event_router.rb:114:emit_stream: /home/daipom/work/fluentd/fluentd/lib/fluent/event_router.rb:106:in `emit'
2025-06-23 18:16:46 +0900 [warn]: #0 fluent/event_router.rb:114:emit_stream: /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:115:in `block in emit'
2025-06-23 18:16:46 +0900 [warn]: #0 fluent/event_router.rb:114:emit_stream: /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:115:in `times'
2025-06-23 18:16:46 +0900 [warn]: #0 fluent/event_router.rb:114:emit_stream: /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:115:in `emit'
2025-06-23 18:16:46 +0900 [warn]: #0 fluent/event_router.rb:114:emit_stream: /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:100:in `run'
2025-06-23 18:16:46 +0900 [warn]: #0 fluent/event_router.rb:114:emit_stream: /home/daipom/work/fluentd/fluentd/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
```
With `forced_stacktrace_level info`:
```
2025-06-23 18:13:59 +0900 [warn]: #0 fluent/log.rb:392:warn: emit transaction failed: error_class=RuntimeError error="failed to flush" location="/home/daipom/work/fluentd/fluentd/lib/fluent/plugin/out_null.rb:54:in `process'" tag="test.fail"
2025-06-23 18:13:59 +0900 [info]: #0 fluent/event_router.rb:114:emit_stream: /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/out_null.rb:54:in `process'
2025-06-23 18:13:59 +0900 [info]: #0 fluent/event_router.rb:114:emit_stream: /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:865:in `emit_sync'
2025-06-23 18:13:59 +0900 [info]: #0 fluent/event_router.rb:114:emit_stream: /home/daipom/work/fluentd/fluentd/lib/fluent/event_router.rb:115:in `emit_stream'
2025-06-23 18:13:59 +0900 [info]: #0 fluent/event_router.rb:114:emit_stream: /home/daipom/work/fluentd/fluentd/lib/fluent/event_router.rb:106:in `emit'
2025-06-23 18:13:59 +0900 [info]: #0 fluent/event_router.rb:114:emit_stream: /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:115:in `block in emit'
2025-06-23 18:13:59 +0900 [info]: #0 fluent/event_router.rb:114:emit_stream: /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:115:in `times'
2025-06-23 18:13:59 +0900 [info]: #0 fluent/event_router.rb:114:emit_stream: /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:115:in `emit'
2025-06-23 18:13:59 +0900 [info]: #0 fluent/event_router.rb:114:emit_stream: /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:100:in `run'
2025-06-23 18:13:59 +0900 [info]: #0 fluent/event_router.rb:114:emit_stream: /home/daipom/work/fluentd/fluentd/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
```
Signed-off-by: Daijiro Fukuda <[email protected]>
Signed-off-by: Daijiro Fukuda <[email protected]> Co-authored-by: Kentaro Hayashi <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://docs.fluentd.org/deployment/system-config#less-than-log-greater-than-section