Skip to content

docs: add Azure Blob service principal authentication #2592

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Vaibhav-C-S wants to merge 5 commits into
base: master
Choose a base branch
from
+47 −2
Open

docs: add Azure Blob service principal authentication #2592

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
17d6042
docs: add Azure Blob service principal authentication
Jun 1, 2026
373748e
add example
Jun 1, 2026
4a0ea41
docs: address review comments
Jun 1, 2026
da34f18
Merge branch 'master' into azure-blob-service-principal-docs
eschabell Jun 27, 2026
7687de1
docs: pipeline: outputs: azure_blob: wrap terms in backticks to clear…
eschabell Jun 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 47 additions & 2 deletions pipeline/outputs/azure_blob.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,16 +23,18 @@ Fluent Bit exposes the following configuration properties.
| Key | Description | Default |
| :------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :---------------------------- |
| `account_name` | Azure Storage account name. | _none_ |
| `auth_type` | Specify the type to authenticate against the service. Supported values: `key`, `sas`. | `key` |
| `auth_type` | Specify the type to authenticate against the service. Supported values: `key`, `sas`, `service_principal`. | `key` |
| `auto_create_container` | If `container_name` doesn't exist in the remote service, enabling this option handles the exception and auto-creates the container. | `true` |
| `azure_blob_buffer_key` | Set the Azure Blob buffer key which needs to be specified when using multiple instances of Azure Blob output plugin and buffering is enabled. | `key` |
| `blob_type` | Specify the desired blob type. Supported values: `appendblob`, `blockblob`. | `appendblob` |
| `blob_uri_length` | Set the length of the generated blob URI used when creating and uploading objects to Azure Blob Storage. | `64` |
| `buffer_dir` | Specifies the location of directory where the buffered data will be stored. | `/tmp/fluent-bit/azure-blob/` |
| `buffer_file_delete_early` | Whether to delete the buffered file early after successful blob creation. | `false` |
| `buffering_enabled` | Enable buffering into disk before ingesting into Azure Blob. This option requires `blob_type` to be set to `blockblob`. It isn't compatible when `blob_type` = `appendblob`. Fluent Bit returns a configuration error and fails to start. | `false` |
| `client_id` | Microsoft `Entra` ID application/client ID. Mandatory when `auth_type` is `service_principal`. | _none_ |
| `client_secret` | Microsoft `Entra` ID client secret. Mandatory when `auth_type` is `service_principal`. | _none_ |
| `compress` | Sets payload compression in network transfer. Supported values: `gzip`, `zstd`. | _none_ |
| `compress_blob` | Enables compression in the final `blockblob` file. When enabled without `compress`, it uses GZIP; if `compress` is also set, it inherits that codec. This option isn't compatible when `blob_type` = `appendblob`. Fluent Bit returns a configuration error and fails to start. | `false` |
| `compress_blob` | Enables compression in the final `blockblob` file. When enabled without `compress`, it uses GZIP; if `compress` is also set, it inherits that `codec`. This option isn't compatible when `blob_type` = `appendblob`. Fluent Bit returns a configuration error and fails to start. | `false` |
| `configuration_endpoint_bearer_token` | Bearer token for the configuration endpoint. | _none_ |
| `configuration_endpoint_password` | Basic authentication password for the configuration endpoint. | _none_ |
| `configuration_endpoint_url` | Configuration endpoint URL. | _none_ |
Expand All @@ -52,6 +54,7 @@ Fluent Bit exposes the following configuration properties.
| `scheduler_max_retries` | Maximum number of retries for the scheduler send blob. | `3` |
| `shared_key` | Specify the Azure Storage Shared Key to authenticate against the service. This configuration property is mandatory when `auth_type` is `key`. | _none_ |
| `store_dir_limit_size` | Set the max size of the buffer directory. | `8G` |
| `tenant_id` | Microsoft `Entra` ID tenant ID. Mandatory when `auth_type` is `service_principal`. | _none_ |
| `tls` | Enable or disable TLS encryption. Azure service requires this to be set to `on`. | `off` |
| `unify_tag` | Whether to create a single buffer file when buffering mode is enabled. | `false` |
| `upload_file_size` | Specifies the size of files to be uploaded in MB. | `200M` |
Expand Down Expand Up @@ -156,6 +159,48 @@ After you run the configuration file, you will be able to query the data using t

![Azure Blob](../../.gitbook/assets/azure_blob.png)

### Configuration with service principal authentication

To authenticate with Microsoft `Entra` ID service principal credentials, set `auth_type` to `service_principal` and configure `tenant_id`, `client_id`, and `client_secret`. The service principal must have permission to write to the target storage account or container, such as the `Storage Blob Data Contributor` role. Service principal authentication requires `tls` to be set to `on` to ensure secure communication.

{% tabs %}
{% tab title="fluent-bit.yaml" %}

```yaml
pipeline:
outputs:
- name: azure_blob
match: "*"
account_name: YOUR_ACCOUNT_NAME
container_name: logs
blob_type: blockblob
auth_type: service_principal
tenant_id: YOUR_TENANT_ID
client_id: YOUR_CLIENT_ID
client_secret: YOUR_CLIENT_SECRET
tls: on
```

{% endtab %}
{% tab title="fluent-bit.conf" %}

```text
[OUTPUT]
Name azure_blob
Match *
Account_Name YOUR_ACCOUNT_NAME
Container_Name logs
Blob_Type blockblob
Auth_Type service_principal
Tenant_Id YOUR_TENANT_ID
Client_Id YOUR_CLIENT_ID
Client_Secret YOUR_CLIENT_SECRET
Tls on
```

{% endtab %}
{% endtabs %}

### Configuring and using Azure Emulator: Azurite

#### Install and run Azurite
Expand Down