Skip to content

Commit 889c23d

Browse files
wenchajunwenchajun
committed
update documents
Signed-off-by: chengdehao <[email protected]>
1 parent b67703f commit 889c23d

19 files changed

+127
-105
lines changed

‎README.md

+15-3
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,9 @@ Once installed, the Fluent Operator provides the following features:
4949
- [Fluentd](#fluentd-2)
5050
- [Best Practice](#best-practice)
5151
- [Plugin Grouping](#plugin-grouping)
52-
- [Custom Parser](#custom-parser)
52+
- [Monitoring](#monitoring)
53+
- [Custom Parser](#custom-parser)
54+
- [Misc](#misc)
5355
- [Roadmap](#roadmap)
5456
- [Development](#development)
5557
- [Requirements](#requirements)
@@ -248,9 +250,19 @@ For more info on various use cases of Fluent Operator Fluentd CRDs, you can refe
248250

249251
### Plugin Grouping
250252

251-
For fluenbit,input, filter, and output plugins are connected by label selectors. For input and output plugins, always create `ClusterInput` or `ClusterOutput` CRs for every plugin. Don't aggregate multiple inputs or outputs into one `ClusterInput` or `ClusterOutput` object, except you have a good reason to do so. Take the demo `logging stack` for example, we have one yaml file for each output.
253+
[Here](docs/best-practice/plugin-grouping.md) you can find the plugin group information.
252254

253-
However, for filter plugins, if you want a filter chain, the order of filters matters. You need to organize multiple filters into an array as the demo [logging stack](manifests/logging-stack/filter-kubernetes.yaml) suggests.For more info on various use cases of Fluent Operator Fluentd CRDs, you can refer to [Fluent-Operator-Walkthrough](https://github.com/kubesphere-sigs/fluent-operator-walkthrough#fluent-bit--fluentd-mode).
255+
### Monitoring
256+
257+
[Here](docs/best-practice/monitoring.md) you can use the built-in HTTP Server in Fluent Bit.
258+
259+
### Custom Parser
260+
261+
[Here](docs/best-practice/custom-parser.md) you can customize parser in Fluent Bit.
262+
263+
### Misc
264+
265+
If you want to learn more about Fluent-Operator, please refer to the [misc](docs/best-practice/misc.md).
254266

255267
## Roadmap
256268

‎docs/user-guides/collect-systemd-logs.md ‎docs/best-practice/collect-systemd-logs.md

File renamed without changes.

‎docs/best-practice/custom-parser.md

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# Custom Parser
2+
3+
To enable parsers, you must set the value of `FluentBitConfig.Spec.Service.ParsersFile` to `parsers.conf`. Your custom parsers will be included into the built-in parser config via `@INCLUDE /fluent-bit/config/parsers.conf`. Note that the parsers.conf contains a few built-in parsers, for example, docker. Read [parsers.conf](https://github.com/kubesphere/fluentbit-operator/blob/master/conf/parsers.conf) for more information.
4+
5+
Check out the demo in the folder `/manifests/regex-parser` for how to use a custom regex parser.
6+

‎docs/user-guides/forwarding-logs-via-http/Dockerfile ‎docs/best-practice/forwarding-logs-via-http/Dockerfile

File renamed without changes.

‎docs/user-guides/forwarding-logs-via-http/README.md ‎docs/best-practice/forwarding-logs-via-http/README.md

File renamed without changes.

‎docs/user-guides/forwarding-logs-via-http/deploy/filter-grep.yaml ‎docs/best-practice/forwarding-logs-via-http/deploy/filter-grep.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
apiVersion: fluentbit.fluent.io/v1alpha2
2-
kind: Filter
2+
kind: ClusterFilter
33
metadata:
44
name: grep
55
labels:

‎docs/user-guides/forwarding-logs-via-http/deploy/fluentbit-fluentBit.yaml ‎docs/best-practice/forwarding-logs-via-http/deploy/fluentbit-fluentBit.yaml

File renamed without changes.

‎docs/user-guides/forwarding-logs-via-http/deploy/fluentbitconfig-fluentBitConfig.yaml ‎docs/best-practice/forwarding-logs-via-http/deploy/fluentbitconfig-fluentBitConfig.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
apiVersion: fluentbit.fluent.io/v1alpha2
2-
kind: FluentBitConfig
2+
kind: ClusterFluentBitConfig
33
metadata:
44
name: fluent-bit-config
55
spec:

‎docs/user-guides/forwarding-logs-via-http/deploy/input-tail.yaml ‎docs/best-practice/forwarding-logs-via-http/deploy/input-tail.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
apiVersion: fluentbit.fluent.io/v1alpha2
2-
kind: Input
2+
kind: ClusterInput
33
metadata:
44
name: tail
55
labels:

‎docs/user-guides/forwarding-logs-via-http/deploy/log-generator.yaml ‎docs/best-practice/forwarding-logs-via-http/deploy/log-generator.yaml

File renamed without changes.

‎docs/user-guides/forwarding-logs-via-http/deploy/output-http.yaml ‎docs/best-practice/forwarding-logs-via-http/deploy/output-http.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
apiVersion: fluentbit.fluent.io/v1alpha2
2-
kind: Output
2+
kind: ClusterOutput
33
metadata:
44
name: http
55
labels:

‎docs/user-guides/forwarding-logs-via-http/kubesphere/filter-grep.yaml ‎docs/best-practice/forwarding-logs-via-http/kubesphere/filter-grep.yaml

File renamed without changes.

‎docs/user-guides/forwarding-logs-via-http/kubesphere/input-tail.yaml ‎docs/best-practice/forwarding-logs-via-http/kubesphere/input-tail.yaml

File renamed without changes.

‎docs/user-guides/forwarding-logs-via-http/kubesphere/output-http.yaml ‎docs/best-practice/forwarding-logs-via-http/kubesphere/output-http.yaml

File renamed without changes.

‎docs/user-guides/forwarding-logs-via-http/main.go ‎docs/best-practice/forwarding-logs-via-http/main.go

File renamed without changes.

‎docs/best-practice/misc.md

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Misc
2+
# Path Convention
3+
4+
Path to file in Fluent Bit config should be well regulated. Fluent Bit Operator adopts the following convention internally.
5+
6+
| Dir Path | Description |
7+
| --------------------------------- | ------------------------------------------------------------ |
8+
| /fluent-bit/tail | Stores tail related files, eg. file tracking db. Using [fluentbit.spec.positionDB](https://github.com/fluent/fluent-operator/blob/master/docs/fluentbit.md#fluentbitspec) will mount a file `pos.db` under this dir by default. |
9+
| /fluent-bit/secrets/{secret_name} | Stores secrets, eg. TLS files. Specify secrets to mount in [fluentbit.spec.secrets](https://github.com/fluent/fluent-operator/blob/master/docs/fluentbit.md#fluentbitspec), then you have access. |
10+
| /fluent-bit/config | Stores the main config file and user-defined parser config file. |
11+
12+
> Note that ServiceAccount files are mounted at `/var/run/secrets/kubernetes.io/serviceaccount`.
13+
14+
15+

‎docs/best-practice/monitoring.md

+77
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# Monitoring
2+
3+
Fluent Bit comes with a built-in HTTP Server. According to the official [documentation](https://docs.fluentbit.io/manual/administration/monitoring) of fluentbit You can enable this by enabling the HTTP server from the fluent bit configuration file:
4+
5+
```conf
6+
[SERVICE]
7+
HTTP_Server On
8+
HTTP_Listen 0.0.0.0
9+
HTTP_PORT 2020
10+
```
11+
12+
When you use the fluent-operator, You can enable this from `FluentBitConfig` manifest. Example is below:
13+
14+
```yaml
15+
apiVersion: fluentbit.fluent.io/v1alpha2
16+
kind: ClusterFluentBitConfig
17+
metadata:
18+
name: fluent-bit-config
19+
labels:
20+
app.kubernetes.io/name: fluent-bit
21+
spec:
22+
filterSelector:
23+
matchLabels:
24+
fluentbit.fluent.io/enabled: 'true'
25+
inputSelector:
26+
matchLabels:
27+
fluentbit.fluent.io/enabled: 'true'
28+
outputSelector:
29+
matchLabels:
30+
fluentbit.fluent.io/enabled: 'true'
31+
service:
32+
httpListen: 0.0.0.0
33+
httpPort: 2020
34+
httpServer: true
35+
parsersFile: parsers.conf
36+
37+
```
38+
39+
Once HTTP server is enabled, you should be able to get the information:
40+
41+
```bash
42+
curl <podIP>:2020 | jq .
43+
44+
{
45+
"fluent-bit": {
46+
"version": "1.8.3",
47+
"edition": "Community",
48+
"flags": [
49+
"FLB_HAVE_PARSER",
50+
"FLB_HAVE_RECORD_ACCESSOR",
51+
"FLB_HAVE_STREAM_PROCESSOR",
52+
"FLB_HAVE_TLS",
53+
"FLB_HAVE_OPENSSL",
54+
"FLB_HAVE_AWS",
55+
"FLB_HAVE_SIGNV4",
56+
"FLB_HAVE_SQLDB",
57+
"FLB_HAVE_METRICS",
58+
"FLB_HAVE_HTTP_SERVER",
59+
"FLB_HAVE_SYSTEMD",
60+
"FLB_HAVE_FORK",
61+
"FLB_HAVE_TIMESPEC_GET",
62+
"FLB_HAVE_GMTOFF",
63+
"FLB_HAVE_UNIX_SOCKET",
64+
"FLB_HAVE_PROXY_GO",
65+
"FLB_HAVE_JEMALLOC",
66+
"FLB_HAVE_LIBBACKTRACE",
67+
"FLB_HAVE_REGEX",
68+
"FLB_HAVE_UTF8_ENCODER",
69+
"FLB_HAVE_LUAJIT",
70+
"FLB_HAVE_C_TLS",
71+
"FLB_HAVE_ACCEPT4",
72+
"FLB_HAVE_INOTIFY"
73+
]
74+
}
75+
}
76+
```
77+

‎docs/best-practice/plugin-grouping.md

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
2+
3+
# Plugin Grouping
4+
5+
For fluenbit,input, filter, and output plugins are connected by label selectors. For input and output plugins, always create `ClusterInput` or `ClusterOutput` CRs for every plugin. Don't aggregate multiple inputs or outputs into one `ClusterInput` or `ClusterOutput` object, except you have a good reason to do so. Take the demo `logging stack` for example, we have one yaml file for each output.
6+
7+
However, for filter plugins, if you want a filter chain, the order of filters matters. You need to organize multiple filters into an array as the demo [logging stack](https://github.com/fluent/fluent-operator/blob/master/manifests/logging-stack/filter-kubernetes.yaml) suggests.For more info on various use cases of Fluent Operator Fluentd CRDs, you can refer to [Fluent-Operator-Walkthrough](https://github.com/kubesphere-sigs/fluent-operator-walkthrough#fluent-bit--fluentd-mode).
8+
9+
10+

‎docs/fluentbit.md

-98
Original file line numberDiff line numberDiff line change
@@ -325,101 +325,3 @@ ParserSpec defines the desired state of ClusterParser
325325
| parsersFile | Optional 'parsers' config file (can be multiple) | string |
326326

327327
[Back to TOC](#table-of-contents)
328-
# Monitoring
329-
330-
Fluent Bit comes with a built-in HTTP Server. According to the official [documentation](https://docs.fluentbit.io/manual/administration/monitoring) of fluentbit You can enable this by enabling the HTTP server from the fluent bit configuration file:
331-
332-
```conf
333-
[SERVICE]
334-
HTTP_Server On
335-
HTTP_Listen 0.0.0.0
336-
HTTP_PORT 2020
337-
```
338-
339-
When you use the fluent-operator, You can enable this from `FluentBitConfig` manifest. Example is below:
340-
341-
```yaml
342-
apiVersion: fluentbit.fluent.io/v1alpha2
343-
kind: ClusterFluentBitConfig
344-
metadata:
345-
name: fluent-bit-config
346-
labels:
347-
app.kubernetes.io/name: fluent-bit
348-
spec:
349-
filterSelector:
350-
matchLabels:
351-
fluentbit.fluent.io/enabled: 'true'
352-
inputSelector:
353-
matchLabels:
354-
fluentbit.fluent.io/enabled: 'true'
355-
outputSelector:
356-
matchLabels:
357-
fluentbit.fluent.io/enabled: 'true'
358-
service:
359-
httpListen: 0.0.0.0
360-
httpPort: 2020
361-
httpServer: true
362-
parsersFile: parsers.conf
363-
364-
```
365-
366-
Once HTTP server is enabled, you should be able to get the information:
367-
368-
```bash
369-
curl <podIP>:2020 | jq .
370-
371-
{
372-
"fluent-bit": {
373-
"version": "1.8.3",
374-
"edition": "Community",
375-
"flags": [
376-
"FLB_HAVE_PARSER",
377-
"FLB_HAVE_RECORD_ACCESSOR",
378-
"FLB_HAVE_STREAM_PROCESSOR",
379-
"FLB_HAVE_TLS",
380-
"FLB_HAVE_OPENSSL",
381-
"FLB_HAVE_AWS",
382-
"FLB_HAVE_SIGNV4",
383-
"FLB_HAVE_SQLDB",
384-
"FLB_HAVE_METRICS",
385-
"FLB_HAVE_HTTP_SERVER",
386-
"FLB_HAVE_SYSTEMD",
387-
"FLB_HAVE_FORK",
388-
"FLB_HAVE_TIMESPEC_GET",
389-
"FLB_HAVE_GMTOFF",
390-
"FLB_HAVE_UNIX_SOCKET",
391-
"FLB_HAVE_PROXY_GO",
392-
"FLB_HAVE_JEMALLOC",
393-
"FLB_HAVE_LIBBACKTRACE",
394-
"FLB_HAVE_REGEX",
395-
"FLB_HAVE_UTF8_ENCODER",
396-
"FLB_HAVE_LUAJIT",
397-
"FLB_HAVE_C_TLS",
398-
"FLB_HAVE_ACCEPT4",
399-
"FLB_HAVE_INOTIFY"
400-
]
401-
}
402-
}
403-
```
404-
405-
[Back to TOC](#table-of-contents)
406-
# Path Convention
407-
408-
Path to file in Fluent Bit config should be well regulated. Fluent Bit Operator adopts the following convention internally.
409-
410-
| Dir Path | Description |
411-
| --------------------------------- | ------------------------------------------------------------ |
412-
| /fluent-bit/tail | Stores tail related files, eg. file tracking db. Using [fluentbit.spec.positionDB](docs/fluentbit.md#fluentbitspec) will mount a file `pos.db` under this dir by default. |
413-
| /fluent-bit/secrets/{secret_name} | Stores secrets, eg. TLS files. Specify secrets to mount in [fluentbit.spec.secrets](docs/fluentbit.md#fluentbitspec), then you have access. |
414-
| /fluent-bit/config | Stores the main config file and user-defined parser config file. |
415-
416-
> Note that ServiceAccount files are mounted at `/var/run/secrets/kubernetes.io/serviceaccount`.
417-
418-
[Back to TOC](#table-of-contents)
419-
# Custom Parser
420-
421-
To enable parsers, you must set the value of `FluentBitConfig.Spec.Service.ParsersFile` to `parsers.conf`. Your custom parsers will be included into the built-in parser config via `@INCLUDE /fluent-bit/config/parsers.conf`. Note that the parsers.conf contains a few built-in parsers, for example, docker. Read [parsers.conf](https://github.com/kubesphere/fluentbit-operator/blob/master/conf/parsers.conf) for more information.
422-
423-
Check out the demo in the folder `/manifests/regex-parser` for how to use a custom regex parser.
424-
425-
[Back to TOC](#table-of-contents)

0 commit comments

Comments
 (0)