s3 output Buffer Full Even with buffer path is empty too

[smiley-ci]

Bug Report

S3 Counter Underflow Error

Ran with fluent-bit 5.0.7 and 5.0.9 both returning the error in sometime though buffer path not filled

[2026/07/23 17:35:18.363] [ warn] [output:s3:s3.0] Could not buffer chunk. Data order preservation will be compromised
[2026/07/23 17:35:18.363] [error] [output:s3:s3.0] Buffer is full: current_buffer_size=18446744073684963328, new_data=9272, store_dir_limit_size=2000000000 bytes

We have added below parameters to get the retried chunks get deleted post retry exceeded we are getting buffer full error though file path is empty

retry_limit         5 
retry_exhausted_action  delete

• Version used: 5.0.7 ,5.0.9

[SERVICE]
Flush 120
Log_Level info
Daemon off
Parsers_File parsers.conf
HTTP_Server ${HTTP_SERVER}
HTTP_Listen 0.0.0.0
HTTP_Port ${HTTP_PORT}
storage.path /var/fluent-bit/state/flb-storage/
storage.sync normal
storage.checksum off
storage.backlog.mem_limit 50M
storage.max_chunks_up 512
[INPUT]
Name fluentbit_metrics
Tag fluentbit_metrics
scrape_interval 30
[INPUT]
Name tail
Tag kubernetes.logs.*
Path /var/log/containers/.log
Docker_Mode On
Docker_Mode_Flush 5
Docker_Mode_Parser container_firstline
Parser docker
DB /var/fluent-bit/state/flb_container.db
Skip_Long_Lines On
Refresh_Interval 5
Rotate_Wait 30
storage.type filesystem
Read_from_Head false
[FILTER]
Name kubernetes
Match kubernetes.logs.
tls.verify Off
Kube_URL https://kubernetes.default.svc.cluster.local:443
Kube_CA_File /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
Kube_Token_File /var/run/secrets/kubernetes.io/serviceaccount/token
Kube_Tag_Prefix kubernetes.logs.var.log.containers
Merge_Log On
Labels Off
Annotations Off
Buffer_Size 0
Use_Kubelet true
Kubelet_Port 10250
[FILTER]
Name nest
Match kubernetes.logs.*
Operation lift
Nested_under kubernetes
[FILTER]
Name modify
Match kubernetes.logs.*
Add tag docker.all
[OUTPUT]
Name s3
Match_regex ^(kubernetes.logs.*)$
Bucket ${S3_BUCKET}
Region us-east-1
endpoint ${S3_ENDPOINT}
compression gzip
s3_key_format /logs/v1_%Y%m%d%H%M%S_$UUID.gz
Upload_chunk_size 50M
static_file_path On
use_put_object Off
store_dir /var/log/fluentbit/log/s3/
store_dir_limit_size 2GB
retry_limit 5
retry_exhausted_action delete
[OUTPUT]
Name prometheus_exporter
match fluentbit_metrics
host ::
port 2022

BufferPath:

/var/log/fluentbit/log/s3$ ls -lhtr
var/log/fluentbit/log/s3$ ls -lhtr
total 16K
drwxr-xr-x 2 root root 4.0K Jul 17 17:07 quarantine
drwxr-xr-x 2 root root 4.0K Jul 17 17:07 multipart_upload_metadata
drwxr-xr-x 2 root root 4.0K Jul 20 15:09 2026-07-20T15:09:42
drwxr-xr-x 2 root root 4.0K Jul 20 15:09 2026-07-17T17:07:11

[Holtee]

This looks like the same defect analyzed in #12164 (root cause + proposed fix): set_files_context() assigns startup-loaded store files a size that is never credited to current_buffer_size, so the startup resend's delete underflows the size_t and the output permanently rejects data. Your wrapped value 18446744073684963328 = 2^64 − ~24.6 MB — the leftover bytes subtracted at startup. Introduced in v5.0.7 (the assignment is absent ≤ v5.0.6). A workaround until it's fixed: avoid restarting with leftover files in store_dir, or downgrade below 5.0.7.

[smiley-ci]

But version less than v5.0.6 does not have retry_exhausted_action delete. we are getting the buffer path full once retries exceeded we want to delete them instead of get them stored in the buffer path