Skip to content
This repository has been archived by the owner on May 17, 2019. It is now read-only.

Commit

Permalink
Merge pull request #432 from fujimotos/sf/explain-exponential-backoff
Browse files Browse the repository at this point in the history
docs: Add a section to explain how exponential backoff works.
  • Loading branch information
fujimotos authored Jan 29, 2018
2 parents 896ecfe + 0c6b8eb commit 91d724b
Showing 1 changed file with 21 additions and 0 deletions.
21 changes: 21 additions & 0 deletions docs/v1.0/buffer-plugin-overview.txt
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,27 @@ All chunks on stage are writable, and events will be appended into these chunks.

Buffer plugins control size limits of each chunks, and size limits of whole buffer plugin. Output plugin referres these limits, and controls flushing and retrying.

## Control Retry Behaviour

A chunk can fail to be written out to the destination for a number of reasons. The network can go down, or the traffic volumes can exceed the capacity of the destination node. To handle such common failures gracefully, buffer plugins are equipped with a built-in retry mechanism.

### How exponential backoff works

By default, Fluentd increases the wait interval exponentially for each retry attempt. For example, assuming that the initial wait interval is set to 1 second and the exponential factor is 2, each attempt occurs at the following time points:

1 2 4 8 16
x-x---x-------x---------------x-------------------------
│ │ │ │ └─ 4th retry (wait = 8s)
│ │ │ └───────────────── 3th retry (wait = 4s)
│ │ └───────────────────────── 2th retry (wait = 2s)
│ └───────────────────────────── 1th retry (wait = 1s)
└─────────────────────────────── FAIL

In practice, Fluentd "flavours" this algorithm in a few aspects:

* Wait intervals are **randomized** by default. That is, Fluentd diversifies the wait interval by multiplying by a randomly-chosen number between 0.875 and 1.125. You can turn off this behaviour by setting `retry_randomize` to false.
* Wait intervals *can* be **capped** to a certain limit. For example, if you set `retry_max_interval` to 5 seconds in the example above, the 4th retry will wait for 5 seconds, instead of 8 seconds.

## Parameters

* [Common Parameters](plugin-common-parameters)
Expand Down

0 comments on commit 91d724b

Please sign in to comment.