Skip to content

Add fields to AzureBlob output plugin #1727

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Oct 22, 2025
Merged

Add fields to AzureBlob output plugin #1727

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c1356c2
Add fields to AzureBlob output plugin
BHYub Oct 14, 2025
1d73383
Merge branch 'master' into add-fields-to-azure-blob-output-plugin
marcofranssen Oct 22, 2025
3a89e6f
Update docs
marcofranssen Oct 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions apis/fluentbit/v1alpha2/plugins/output/azure_blob_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,17 @@ type AzureBlob struct {
EmulatorMode string `json:"emulatorMode,omitempty"`
// HTTP Service of the endpoint (if using EmulatorMode)
Endpoint string `json:"endpoint,omitempty"`
// Optional: Enables GZIP compression in the final blockblob file. This option isn't compatible when blob_type = appendblob.
// +kubebuilder:validation:Enum:=on;off
CompressBlob string `json:"compressBlob,omitempty"`
// Enable buffering into disk before ingesting into Azure Blob.
BufferingEnabled *bool `json:"bufferingEnabled,omitempty"`
// Specifies the size of files to be uploaded in MB. Defaults to 200M.
// +kubebuilder:default:="200M"
UploadFileSize string `json:"uploadFileSize,omitempty"`
// Optional. Specify a timeout for uploads. Fluent Bit will start ingesting buffer files which have been created more than x minutes ago and haven't reached upload_file_size limit yet. Defaults to 30m.
// +kubebuilder:default:="30m"
UploadTimeout string `json:"uploadTimeout,omitempty"`
// Enable/Disable TLS Encryption. Azure services require TLS to be enabled.
*plugins.TLS `json:"tls,omitempty"`
// Include fluentbit networking options for this output-plugin
Expand Down Expand Up @@ -67,6 +78,10 @@ func (o *AzureBlob) Params(sl plugins.SecretLoader) (*params.KVs, error) {
plugins.InsertKVString(kvs, "container_name", o.ContainerName)
plugins.InsertKVString(kvs, "blob_type", o.BlobType)
plugins.InsertKVString(kvs, "auto_create_container", o.AutoCreateContainer)
plugins.InsertKVString(kvs, "compress_blob", o.CompressBlob)
plugins.InsertKVField(kvs, "buffering_enabled", o.BufferingEnabled)
plugins.InsertKVString(kvs, "upload_file_size", o.UploadFileSize)
plugins.InsertKVString(kvs, "upload_timeout", o.UploadTimeout)
plugins.InsertKVString(kvs, "path", o.Path)
plugins.InsertKVString(kvs, "emulator_mode", o.EmulatorMode)
plugins.InsertKVString(kvs, "endpoint", o.Endpoint)
Expand Down
5 changes: 5 additions & 0 deletions apis/fluentbit/v1alpha2/plugins/output/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

24 changes: 24 additions & 0 deletions charts/fluent-operator/charts/fluent-bit-crds/crds/fluentbit.fluent.io_clusteroutputs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,18 @@ spec:
- appendblob
- blockblob
type: string
bufferingEnabled:
description: Enable buffering into disk before ingesting into
Azure Blob.
type: boolean
compressBlob:
description: 'Optional: Enables GZIP compression in the final
blockblob file. This option isn''t compatible when blob_type
= appendblob.'
enum:
- "on"
- "off"
type: string
containerName:
description: Name of the container that will contain the blobs
type: string
Expand Down Expand Up @@ -240,6 +252,18 @@ spec:
description: Hostname to be used for TLS SNI extension
type: string
type: object
uploadFileSize:
default: 200M
description: Specifies the size of files to be uploaded in MB.
Defaults to 200M.
type: string
uploadTimeout:
default: 30m
description: Optional. Specify a timeout for uploads. Fluent Bit
will start ingesting buffer files which have been created more
than x minutes ago and haven't reached upload_file_size limit
yet. Defaults to 30m.
type: string
required:
- accountName
- containerName
Expand Down
24 changes: 24 additions & 0 deletions charts/fluent-operator/charts/fluent-bit-crds/crds/fluentbit.fluent.io_outputs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,18 @@ spec:
- appendblob
- blockblob
type: string
bufferingEnabled:
description: Enable buffering into disk before ingesting into
Azure Blob.
type: boolean
compressBlob:
description: 'Optional: Enables GZIP compression in the final
blockblob file. This option isn''t compatible when blob_type
= appendblob.'
enum:
- "on"
- "off"
type: string
containerName:
description: Name of the container that will contain the blobs
type: string
Expand Down Expand Up @@ -240,6 +252,18 @@ spec:
description: Hostname to be used for TLS SNI extension
type: string
type: object
uploadFileSize:
default: 200M
description: Specifies the size of files to be uploaded in MB.
Defaults to 200M.
type: string
uploadTimeout:
default: 30m
description: Optional. Specify a timeout for uploads. Fluent Bit
will start ingesting buffer files which have been created more
than x minutes ago and haven't reached upload_file_size limit
yet. Defaults to 30m.
type: string
required:
- accountName
- containerName
Expand Down
24 changes: 24 additions & 0 deletions config/crd/bases/fluentbit.fluent.io_clusteroutputs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,18 @@ spec:
- appendblob
- blockblob
type: string
bufferingEnabled:
description: Enable buffering into disk before ingesting into
Azure Blob.
type: boolean
compressBlob:
description: 'Optional: Enables GZIP compression in the final
blockblob file. This option isn''t compatible when blob_type
= appendblob.'
enum:
- "on"
- "off"
type: string
containerName:
description: Name of the container that will contain the blobs
type: string
Expand Down Expand Up @@ -240,6 +252,18 @@ spec:
description: Hostname to be used for TLS SNI extension
type: string
type: object
uploadFileSize:
default: 200M
description: Specifies the size of files to be uploaded in MB.
Defaults to 200M.
type: string
uploadTimeout:
default: 30m
description: Optional. Specify a timeout for uploads. Fluent Bit
will start ingesting buffer files which have been created more
than x minutes ago and haven't reached upload_file_size limit
yet. Defaults to 30m.
type: string
required:
- accountName
- containerName
Expand Down
24 changes: 24 additions & 0 deletions config/crd/bases/fluentbit.fluent.io_outputs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,18 @@ spec:
- appendblob
- blockblob
type: string
bufferingEnabled:
description: Enable buffering into disk before ingesting into
Azure Blob.
type: boolean
compressBlob:
description: 'Optional: Enables GZIP compression in the final
blockblob file. This option isn''t compatible when blob_type
= appendblob.'
enum:
- "on"
- "off"
type: string
containerName:
description: Name of the container that will contain the blobs
type: string
Expand Down Expand Up @@ -240,6 +252,18 @@ spec:
description: Hostname to be used for TLS SNI extension
type: string
type: object
uploadFileSize:
default: 200M
description: Specifies the size of files to be uploaded in MB.
Defaults to 200M.
type: string
uploadTimeout:
default: 30m
description: Optional. Specify a timeout for uploads. Fluent Bit
will start ingesting buffer files which have been created more
than x minutes ago and haven't reached upload_file_size limit
yet. Defaults to 30m.
type: string
required:
- accountName
- containerName
Expand Down
4 changes: 4 additions & 0 deletions docs/plugins/fluentbit/output/azure_blob.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,9 @@ Azure Blob is the Azure Blob output plugin, allows to ingest your records into A
| path | Optional path to store the blobs. | string |
| emulatorMode | Optional toggle to use an Azure emulator | string |
| endpoint | HTTP Service of the endpoint (if using EmulatorMode) | string |
| compressBlob | Optional: Enables GZIP compression in the final blockblob file. This option isn't compatible when blob_type = appendblob. | string |
| bufferingEnabled | Enable buffering into disk before ingesting into Azure Blob. | *bool |
| uploadFileSize | Specifies the size of files to be uploaded in MB. Defaults to 200M. | string |
| uploadTimeout | Optional. Specify a timeout for uploads. Fluent Bit will start ingesting buffer files which have been created more than x minutes ago and haven't reached upload_file_size limit yet. Defaults to 30m. | string |
| tls | Enable/Disable TLS Encryption. Azure services require TLS to be enabled. | *[plugins.TLS](../tls.md) |
| networking | Include fluentbit networking options for this output-plugin | *plugins.Networking |
48 changes: 48 additions & 0 deletions manifests/setup/fluent-operator-crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4187,6 +4187,18 @@ spec:
- appendblob
- blockblob
type: string
bufferingEnabled:
description: Enable buffering into disk before ingesting into
Azure Blob.
type: boolean
compressBlob:
description: 'Optional: Enables GZIP compression in the final
blockblob file. This option isn''t compatible when blob_type
= appendblob.'
enum:
- "on"
- "off"
type: string
containerName:
description: Name of the container that will contain the blobs
type: string
Expand Down Expand Up @@ -4362,6 +4374,18 @@ spec:
description: Hostname to be used for TLS SNI extension
type: string
type: object
uploadFileSize:
default: 200M
description: Specifies the size of files to be uploaded in MB.
Defaults to 200M.
type: string
uploadTimeout:
default: 30m
description: Optional. Specify a timeout for uploads. Fluent Bit
will start ingesting buffer files which have been created more
than x minutes ago and haven't reached upload_file_size limit
yet. Defaults to 30m.
type: string
required:
- accountName
- containerName
Expand Down Expand Up @@ -33213,6 +33237,18 @@ spec:
- appendblob
- blockblob
type: string
bufferingEnabled:
description: Enable buffering into disk before ingesting into
Azure Blob.
type: boolean
compressBlob:
description: 'Optional: Enables GZIP compression in the final
blockblob file. This option isn''t compatible when blob_type
= appendblob.'
enum:
- "on"
- "off"
type: string
containerName:
description: Name of the container that will contain the blobs
type: string
Expand Down Expand Up @@ -33388,6 +33424,18 @@ spec:
description: Hostname to be used for TLS SNI extension
type: string
type: object
uploadFileSize:
default: 200M
description: Specifies the size of files to be uploaded in MB.
Defaults to 200M.
type: string
uploadTimeout:
default: 30m
description: Optional. Specify a timeout for uploads. Fluent Bit
will start ingesting buffer files which have been created more
than x minutes ago and haven't reached upload_file_size limit
yet. Defaults to 30m.
type: string
required:
- accountName
- containerName
Expand Down
48 changes: 48 additions & 0 deletions manifests/setup/setup.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4187,6 +4187,18 @@ spec:
- appendblob
- blockblob
type: string
bufferingEnabled:
description: Enable buffering into disk before ingesting into
Azure Blob.
type: boolean
compressBlob:
description: 'Optional: Enables GZIP compression in the final
blockblob file. This option isn''t compatible when blob_type
= appendblob.'
enum:
- "on"
- "off"
type: string
containerName:
description: Name of the container that will contain the blobs
type: string
Expand Down Expand Up @@ -4362,6 +4374,18 @@ spec:
description: Hostname to be used for TLS SNI extension
type: string
type: object
uploadFileSize:
default: 200M
description: Specifies the size of files to be uploaded in MB.
Defaults to 200M.
type: string
uploadTimeout:
default: 30m
description: Optional. Specify a timeout for uploads. Fluent Bit
will start ingesting buffer files which have been created more
than x minutes ago and haven't reached upload_file_size limit
yet. Defaults to 30m.
type: string
required:
- accountName
- containerName
Expand Down Expand Up @@ -33213,6 +33237,18 @@ spec:
- appendblob
- blockblob
type: string
bufferingEnabled:
description: Enable buffering into disk before ingesting into
Azure Blob.
type: boolean
compressBlob:
description: 'Optional: Enables GZIP compression in the final
blockblob file. This option isn''t compatible when blob_type
= appendblob.'
enum:
- "on"
- "off"
type: string
containerName:
description: Name of the container that will contain the blobs
type: string
Expand Down Expand Up @@ -33388,6 +33424,18 @@ spec:
description: Hostname to be used for TLS SNI extension
type: string
type: object
uploadFileSize:
default: 200M
description: Specifies the size of files to be uploaded in MB.
Defaults to 200M.
type: string
uploadTimeout:
default: 30m
description: Optional. Specify a timeout for uploads. Fluent Bit
will start ingesting buffer files which have been created more
than x minutes ago and haven't reached upload_file_size limit
yet. Defaults to 30m.
type: string
required:
- accountName
- containerName
Expand Down
Loading