Skip to content

Commit 149d575

Browse files
joshuabairdjoshuabaird
authored
Sync helm-chart with upstream. (#560)
Signed-off-by: Josh Baird <[email protected]>
1 parent 5b3161a commit 149d575

File tree

9 files changed

+118
-131
lines changed

9 files changed

+118
-131
lines changed

‎charts/fluent-operator/Chart.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,4 +23,4 @@ dependencies:
2323
- name: fluentd-crds
2424
repository: "file://charts/fluentd-crds"
2525
version: 3.2.0
26-
condition: fluentd.crdsEnable
26+
condition: fluentd.crdsEnable

‎charts/fluent-operator/README.md‎

Lines changed: 16 additions & 57 deletions
Original file line numberDiff line numberDiff line change
@@ -1,76 +1,35 @@
1-
# Fluent Operator Helm Chart
1+
# Fluent Operator Helm chart
22

3-
[Fluent Operator](https://github.com/fluent/fluent-operator) provides great flexibility in building a logging layer based on Fluent Bit and Fluentd.
3+
[Fluent Operator](https://github.com/fluent/fluent-operator/) provides a Kubernetes-native logging pipeline based on Fluent-Bit and Fluentd.
44

5-
## Deploy Fluent Operator with Helm
5+
## Installation
66

7-
> Note: For the helm based install, Helm v3.2.1 or higher is needed.
8-
9-
The Fluent Bit section of the Fluent Operator supports different CRI `docker`, `containerd`, and `CRI-O`.
10-
`containerd` and `CRI-O` use the `CRI Log` format which is different with `docker`, they requires additional parser to parse JSON application logs. You should set different `containerRuntime` depending on your container runtime.
11-
12-
The default runtime is docker, you can choose other runtimes as follows.
13-
14-
If your container runtime is `containerd` or `cri-o`, you can set the `containerRuntime` parameter to `containerd` or `crio`. e.g.
7+
To install or upgrade Fluent Operator using Helm:
158

169
```shell
17-
helm install fluent-operator --create-namespace -n fluent charts/fluent-operator/ --set containerRuntime=containerd
18-
```
10+
export FLUENT_OPERATOR_CONTAINER_RUNTIME="containerd" # or "cri-o", "docker" depending on the container runtime being used (see `values.yaml`)
1911

20-
Install through the online chart link:
21-
22-
```shell
23-
helm install fluent-operator --create-namespace -n fluent https://github.com/fluent/fluent-operator/releases/download/< version >/fluent-operator.tgz
12+
helm repo add fluent https://fluent.github.io/helm-charts
13+
helm upgrade --install fluent-operator fluent/fluent-operator \
14+
--create-namespace \
15+
--set containerRuntime=${FLUENT_OPERATOR_CONTAINER_RUNTIME}
2416
```
2517

26-
> Please replace < version > with a actual version like v1.0.0
18+
By default, all CRDs required for Fluent Operator will be installed. To prevent `helm install` from installing CRDs, you can set `fluent-bit.crdsEnable` or `fluentd.crdsEnable` to `false`.
2719

28-
Fluent Operator CRDs will be installed by default when running a helm install for the chart. But if the CRD already exists, it will be skipped with a warning. So make sure you install the CRDs by yourself if you upgrade your Fluent Operator version.
20+
## Upgrading
2921

30-
> Note: During the upgrade process, if a CRD was previously created using the create operation, an error will occur during the apply operation. Using apply here allows the CRD to be replaced and created in its entirety in a single operation.
22+
Helm [does not manage the lifecycle of CRDs](https://helm.sh/docs/chart_best_practices/custom_resource_definitions/), so if the Fluent Operator CRDs already exist, subsequent
23+
chart upgrades will not add or remove CRDs even if they have changed. During upgrades, users should manually update CRDs:
3124

32-
To replace the CRDs pull the current helm chart:
3325
```
3426
wget https://github.com/fluent/fluent-operator/releases/download/<version>/fluent-operator.tgz
3527
tar -xf fluent-operator.tgz
36-
```
37-
38-
To update the CRDs, run the following command:
39-
40-
```shell
4128
kubectl replace -f fluent-operator/crds
4229
```
4330

44-
## Fluent Operator Walkthrough
45-
46-
For more info on various use cases of Fluent Operator, you can refer to [Fluent-Operator-Walkthrough](https://github.com/kubesphere-sigs/fluent-operator-walkthrough).
47-
48-
## Collect Kubernetes logs
49-
This guide provisions a logging pipeline including the Fluent Bit DaemonSet and its log input/filter/output configurations to collect Kubernetes logs including container logs and kubelet logs.
50-
51-
![logging stack](https://raw.githubusercontent.com/fluent/fluent-operator/master/docs/images/logging-stack.svg)
52-
53-
> Note that you need a running Elasticsearch v5+ cluster to receive log data before start. **Remember to adjust [output-elasticsearch.yaml](https://github.com/fluent/fluent-operator/blob/master/manifests/logging-stack/output-elasticsearch.yaml) to your own es setup**. Kafka and Fluentd outputs are optional and are turned off by default.
54-
55-
#### Deploy the Kubernetes logging pipeline with Helm
56-
57-
You can also deploy the Kubernetes logging pipeline with Helm, just need to set the `Kubernetes` parameter to `true`(default):
58-
59-
```shell
60-
helm upgrade fluent-operator --create-namespace -n fluent charts/fluent-operator/ --set containerRuntime=docker
61-
```
62-
63-
If you want to deploy `fluentd`, just need to set the `fluentd.enable` parameter to `true`.:
31+
## Chart Values
6432

65-
```shell
66-
helm upgrade fluent-operator --create-namespace -n fluent charts/fluent-operator/ --set containerRuntime=docker,fluentd.enable=true
67-
```
68-
69-
Within a couple of minutes, you should observe an index available:
70-
71-
```shell
72-
$ curl localhost:9200/_cat/indices
73-
green open ks-logstash-log-2020.04.26 uwQuoO90TwyigqYRW7MDYQ 1 1 99937 0 31.2mb 31.2mb
7433
```
75-
76-
Success!
34+
helm show values fluent/fluent-operator
35+
```

‎charts/fluent-operator/charts/fluent-bit-crds/Chart.yaml‎

Lines changed: 20 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,23 @@
11
apiVersion: v2
22
name: fluent-bit-crds
3-
description: A Helm chart delivering fluent-bit controller CRDS
4-
keywords:
5-
- logging
6-
- fluent-bit
7-
- operator
3+
description: A Helm chart delivering fluenbt-bit controller CRDS
4+
# A chart can be either an 'application' or a 'library' chart.
5+
#
6+
# Application charts are a collection of templates that can be packaged into versioned archives
7+
# to be deployed.
8+
#
9+
# Library charts provide useful utilities or functions for the chart developer. They're included as
10+
# a dependency of application charts to inject those utilities and functions into the rendering
11+
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
12+
#type: application
13+
14+
# This is the chart version. This version number should be incremented each time you make changes
15+
# to the chart and its templates, including the app version.
16+
# Versions are expected to follow Semantic Versioning (https://semver.org/)
817
version: 3.2.0
9-
appVersion: "3.2.0"
10-
sources:
11-
- https://github.com/fluent/fluent-operator
12-
maintainers:
13-
- name: wenchajun
14-
email: [email protected]
18+
19+
# This is the version number of the application being deployed. This version number should be
20+
# incremented each time you make changes to the application. Versions are not expected to
21+
# follow Semantic Versioning. They should reflect the version the application is using.
22+
# It is recommended to use it with quotes.
23+
appVersion: "3.1.0"

‎charts/fluent-operator/charts/fluent-bit-crds/crds/fluentbit.fluent.io_clusterinputs.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -809,6 +809,10 @@ spec:
809809
a file once is rotated in case some pending data is flushed.
810810
format: int64
811811
type: integer
812+
skipEmptyLines:
813+
description: Skips empty lines in the log file from any further
814+
processing or output.
815+
type: boolean
812816
skipLongLines:
813817
description: |-
814818
When a monitored file reach it buffer capacity due to a very long line (Buffer_Max_Size),

‎charts/fluent-operator/charts/fluent-bit-crds/crds/fluentbit.fluent.io_clusteroutputs.yaml‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -289,6 +289,10 @@ spec:
289289
logType:
290290
description: Name of the event type.
291291
type: string
292+
logTypeKey:
293+
description: Set a record key that will populate 'logtype'. If
294+
the key is found, it will have precedence
295+
type: string
292296
sharedKey:
293297
description: Specify the primary or the secondary client authentication
294298
key
@@ -1747,7 +1751,6 @@ spec:
17471751
type: string
17481752
host:
17491753
description: IP address or hostname of the target InfluxDB service.
1750-
format: ipv6
17511754
type: string
17521755
httpPassword:
17531756
description: Password for user defined in HTTP_User
@@ -1917,8 +1920,8 @@ spec:
19171920
port:
19181921
description: TCP port of the target InfluxDB service.
19191922
format: int32
1920-
maximum: 65536
1921-
minimum: 0
1923+
maximum: 65535
1924+
minimum: 1
19221925
type: integer
19231926
sequenceTag:
19241927
description: The name of the tag whose value is incremented for

‎charts/fluent-operator/charts/fluent-bit-crds/crds/fluentbit.fluent.io_outputs.yaml‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -289,6 +289,10 @@ spec:
289289
logType:
290290
description: Name of the event type.
291291
type: string
292+
logTypeKey:
293+
description: Set a record key that will populate 'logtype'. If
294+
the key is found, it will have precedence
295+
type: string
292296
sharedKey:
293297
description: Specify the primary or the secondary client authentication
294298
key
@@ -1747,7 +1751,6 @@ spec:
17471751
type: string
17481752
host:
17491753
description: IP address or hostname of the target InfluxDB service.
1750-
format: ipv6
17511754
type: string
17521755
httpPassword:
17531756
description: Password for user defined in HTTP_User
@@ -1917,8 +1920,8 @@ spec:
19171920
port:
19181921
description: TCP port of the target InfluxDB service.
19191922
format: int32
1920-
maximum: 65536
1921-
minimum: 0
1923+
maximum: 65535
1924+
minimum: 1
19221925
type: integer
19231926
sequenceTag:
19241927
description: The name of the tag whose value is incremented for

‎charts/fluent-operator/charts/fluentd-crds/Chart.yaml‎

Lines changed: 19 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,23 @@
11
apiVersion: v2
22
name: fluentd-crds
33
description: A Helm chart delivering fluentd controller CRDS
4-
keywords:
5-
- logging
6-
- fluentd
7-
- operator
4+
# A chart can be either an 'application' or a 'library' chart.
5+
#
6+
# Application charts are a collection of templates that can be packaged into versioned archives
7+
# to be deployed.
8+
#
9+
# Library charts provide useful utilities or functions for the chart developer. They're included as
10+
# a dependency of application charts to inject those utilities and functions into the rendering
11+
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
12+
#type: application
13+
14+
# This is the chart version. This version number should be incremented each time you make changes
15+
# to the chart and its templates, including the app version.
16+
# Versions are expected to follow Semantic Versioning (https://semver.org/)
817
version: 3.2.0
9-
appVersion: "3.2.0"
10-
sources:
11-
- https://github.com/fluent/fluent-operator
12-
maintainers:
13-
- name: wenchajun
14-
email: [email protected]
18+
19+
# This is the version number of the application being deployed. This version number should be
20+
# incremented each time you make changes to the application. Versions are not expected to
21+
# follow Semantic Versioning. They should reflect the version the application is using.
22+
# It is recommended to use it with quotes.
23+
appVersion: "3.1.0"

‎charts/fluent-operator/templates/fluentbit-fluentBit.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ spec:
113113
{{- end }}
114114
{{- if .Values.fluentbit.initContainers }}
115115
initContainers:
116-
{{ toYaml .Values.fluentbit.initContainers | indent 4 }}
116+
{{ toYaml .Values.fluentbit.initContainers | indent 4 }}
117117
{{- end }}
118118
{{- end }}
119119
{{- end }}

‎charts/fluent-operator/values.yaml‎

Lines changed: 45 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,9 @@ operator:
2424
memory: 64Mi
2525
container:
2626
repository: "kubesphere/fluent-operator"
27-
tag: "v3.2.0"
27+
tag: "latest"
2828
# If set to false, this will disable the creation of ClusterRole, ClusterRoleBinding,
29-
# Deployment, and ServiceAccount resources to avoid conflicts when deploying multiple instances.
29+
# Deployment, and ServiceAccount resources to avoid conflicts when deploying multiple instances.
3030
enable: true
3131
# nodeSelector configuration for Fluent Operator. Ref: https://kubernetes.io/docs/user-guide/node-selection/
3232
nodeSelector: {}
@@ -86,7 +86,7 @@ fluentbit:
8686
metricRelabelings: []
8787
image:
8888
repository: "ghcr.io/fluent/fluent-operator/fluent-bit"
89-
tag: "3.1.7"
89+
tag: "3.1.8"
9090
# fluentbit resources. If you do want to specify resources, adjust them as necessary
9191
# You can adjust it based on the log volume.
9292
resources:
@@ -125,7 +125,7 @@ fluentbit:
125125
# List of volumes that can be mounted by containers belonging to the pod.
126126
additionalVolumes: []
127127
# Additional rbac rules which will be applied to the fluent-bit clusterrole. Ref: https://kubernetes.io/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding
128-
# NOTE: As fluent-bit is managed by the fluent-operator, fluent-bit can only be granted permissions the operator also has
128+
# NOTE: As fluent-bit is managed by the fluent-operator, fluent-bit can only be granted permissions the operator also has
129129
rbacRules: {}
130130
# Pod volumes to mount into the container's filesystem.
131131
additionalVolumesMounts: []
@@ -203,7 +203,7 @@ fluentbit:
203203
# Use storageType as "filesystem" if you want to use filesystem as the buffering mechanism for tail input.
204204
storageType: memory
205205
pauseOnChunksOverlimit: "off"
206-
# multiline.parser
206+
# multiline.parser
207207
# multilineParser: "docker, cri"
208208
systemd:
209209
enable: true
@@ -287,51 +287,51 @@ fluentbit:
287287
# See https://docs.fluentbit.io/manual/pipeline/outputs/loki
288288
loki:
289289
# Switch for generation of fluentbit loki ClusterOutput (and loki basic auth http user and pass secrets if required)
290-
enable: false # Bool
291-
host: 127.0.0.1 # String
290+
enable: false # Bool
291+
host: 127.0.0.1 # String
292292
port: 3100 # Int
293293
# Either, give http{User,Password},tenantID string values specifying them directly
294294
httpUser: myuser
295295
httpPassword: mypass
296296
tenantID: ''
297297
# Or give {http{User,Password},tenantID} as reference to secrets that you have manually installed into your kubernetes cluster
298-
# httpUser:
299-
# valueFrom:
300-
# secretKeyRef:
301-
# key: value
302-
# name: husersecret
303-
# optional: true
304-
# httpPassword:
305-
# valueFrom:
306-
# secretKeyRef:
307-
# key: value
308-
# name: hpasssecret
309-
# optional: true
310-
# tenantID:
311-
# valueFrom:
312-
# secretKeyRef:
313-
# key: value
314-
# name: tenantsecret
315-
# optional: true
298+
#httpUser:
299+
# valueFrom:
300+
# secretKeyRef:
301+
# key: value
302+
# name: husersecret
303+
# optional: true
304+
#httpPassword:
305+
# valueFrom:
306+
# secretKeyRef:
307+
# key: value
308+
# name: hpasssecret
309+
# optional: true
310+
#tenantID:
311+
# valueFrom:
312+
# secretKeyRef:
313+
# key: value
314+
# name: tenantsecret
315+
# optional: true
316316
#
317-
# To use bearer token auth instead of http basic auth
318-
# bearerToken: ey....
319-
# or with existing secret
320-
# bearerToken:
321-
# valueFrom:
322-
# secretKeyRef:
323-
# key: value
324-
# name: bearerTokenSecret
325-
# optional: true
326-
# labels: [] # String list of <name>=<value>
327-
# labelKeys: [] # String list of <key>
328-
# removeKeys: [] # String list of <key>
329-
# labelMapPath: '' # String, path to file, ex /here/it/is
330-
# dropSingleKey: off
331-
# lineFormat: '' # String
332-
# autoKubernetesLabels: on
333-
# tenantIDKey: # String
334-
# tls: {} # *plugins.TLS fluentbit docs
317+
# To use bearer token auth instead of http basic auth
318+
#bearerToken: ey....
319+
# or with existing secret
320+
#bearerToken:
321+
# valueFrom:
322+
# secretKeyRef:
323+
# key: value
324+
# name: bearerTokenSecret
325+
# optional: true
326+
#labels: [] # String list of <name>=<value>
327+
#labelKeys: [] # String list of <key>
328+
#removeKeys: [] # String list of <key>
329+
#labelMapPath: '' # String, path to file, ex /here/it/is
330+
#dropSingleKey: off
331+
#lineFormat: '' # String
332+
#autoKubernetesLabels: on
333+
#tenantIDKey: # String
334+
#tls: {} # *plugins.TLS fluentbit docs
335335
stackdriver: {}
336336
# You can configure the stackdriver configuration here
337337

@@ -354,7 +354,7 @@ fluentbit:
354354
enable: false
355355
keyContent: log
356356
# emitterMemBufLimit 120 (MB)
357-
emitterMemBufLimit: 120
357+
emitterMemBufLimit: 120
358358
parsers:
359359
- go
360360
- python
@@ -401,7 +401,7 @@ fluentd:
401401
port: 24224
402402
image:
403403
repository: "ghcr.io/fluent/fluent-operator/fluentd"
404-
tag: "v1.17.0"
404+
tag: "v1.17.0-4"
405405
# Numbers of the Fluentd instance
406406
# Applicable when the mode is "collector", and will be ignored when the mode is "agent"
407407
replicas: 1

0 commit comments

Comments
 (0)