home/charts
2
0
Fork 0
mirror of https://github.com/wahyd4/charts.git synced 2026-08-22 03:25:53 +10:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #1558 from alemorcuq/fluentd-improvements

Browse Source 
[bitnami/fluentd] Implement new features
This commit is contained in:
Alejandro Moreno
2019-11-08 13:07:58 +01:00
committed by GitHub
parent ccbf90efb3 ff47b234e5
commit f47429d1e9
14 changed files with 362 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files
bitnami/fluentd/Chart.yaml
+1 -1
View File
@@ -1,6 +1,6 @@
apiVersion: v1
name: fluentd
version: 0.2.0
version: 0.3.0
appVersion: 1.7.4
description: Fluentd is an open source data collector for unified logging layer
keywords:
bitnami/fluentd/README.md
+15
View File
@@ -60,10 +60,18 @@ The following tables lists the configurable parameters of the kibana chart and t
| `nameOverride` | String to partially override fluentd.fullname template with a string (will prepend the release name) | `nil` |
| `fullnameOverride` | String to fully override fluentd.fullname template with a string | `nil` |
| `clusterDomain` | Kubernetes DNS domain name to use | `cluster.local` |
| `forwarder.enabled` | Enable Fluentd aggregator | `true` |
| `forwarder.configFile` | Name of the config file that will be used by Fluentd at launch under the `/opt/bitnami/fluentd/conf` directory | `fluentd.conf` |
| `forwarder.configMap` | Name of the config map that contains the Fluentd configuration files | `nil` |
| `forwarder.extraArgs` | Extra arguments for the Fluentd command line | `nil` |
| `forwarder.extraEnv` | Extra environment variables to pass to the container | `{}` |
| `forwarder.containerPorts` | Ports the forwarder containers will listen on | `Check values.yaml` |
| `forwarder.service.type` | Kubernetes service type (`ClusterIP`, `NodePort`, or `LoadBalancer`) for the forwarders | `ClusterIP` |
| `forwarder.service.ports` | Array containing the forwarder service ports | `Check values.yaml file` |
| `forwarder.service.loadBalancerIP` | loadBalancerIP if service type is `LoadBalancer` | `nil` |
| `forwarder.service.loadBalancerSourceRanges` | Addresses that are allowed when service is LoadBalancer | `[]` |
| `forwarder.service.clusterIP` | Static clusterIP or None for headless services | `nil` |
| `forwarder.service.annotations` | Annotations for the forwarder service | `{}` |
| `forwarder.livenessProbe.enabled` | Enable liveness probes for the forwarder | `true` |
| `forwarder.livenessProbe.initialDelaySeconds` | Delay before liveness probe is initiated | `60` |
| `forwarder.livenessProbe.periodSeconds` | How often to perform the probe | `10` |
@@ -89,6 +97,13 @@ The following tables lists the configurable parameters of the kibana chart and t
| `aggregator.port` | Kubernetes Service port - Fluentd transport port for the aggregators | `24224` |
| `aggregator.extraArgs` | Extra arguments for the Fluentd command line | `nil` |
| `aggregator.extraEnv` | Extra environment variables to pass to the container | `{}` |
| `aggregator.containerPorts` | Ports the aggregator containers will listen on | `Check values.yaml` |
| `aggregator.service.type` | Kubernetes service type (`ClusterIP`, `NodePort`, or `LoadBalancer`) for the aggregators | `ClusterIP` |
| `aggregator.service.ports` | Array containing the aggregator service ports | `Check values.yaml file` |
| `aggregator.service.loadBalancerIP` | loadBalancerIP if service type is `LoadBalancer` | `nil` |
| `aggregator.service.loadBalancerSourceRanges` | Addresses that are allowed when service is LoadBalancer | `[]` |
| `aggregator.service.clusterIP` | Static clusterIP or None for headless services | `nil` |
| `aggregator.service.annotations` | Annotations for the aggregator service | `{}` |
| `aggregator.livenessProbe.enabled` | Enable liveness probes for the aggregator | `true` |
| `aggregator.livenessProbe.initialDelaySeconds` | Delay before liveness probe is initiated | `60` |
| `aggregator.livenessProbe.periodSeconds` | How often to perform the probe | `10` |
bitnami/fluentd/templates/NOTES.txt
+10 -2
View File
@@ -3,15 +3,22 @@
To verify that Fluentd has started, run:
kubectl get all -l "app.kubernetes.io/name={{ include "fluentd.name" . }},app.kubernetes.io/instance={{ .Release.Name }}"
{{ if and .Values.aggregator.enabled (not .Values.aggregator.configMap) }}
{{ if and .Values.aggregator.enabled .Values.forwarder.enabled (not .Values.aggregator.configMap) }}
Logs are captured on each node by the forwarder pods and then sent to the aggregator pods. By default, the aggregator pods send the logs to the standard output.
You can see all the logs by running this command:
kubectl logs -l "app.kubernetes.io/component=aggregator"
You can mount your own configuration files to the aggregators and the forwarders. For example, this is useful if you want to forward the aggregated logs to Elasticsearch or another service.
{{- else if and .Values.aggregator.enabled (not .Values.forwarder.enabled) }}
You have deployed Fluentd in aggregator-only mode. Logs received by the aggregator will be thrown to the standard output by default.
You can see all the logs by running this command:
kubectl logs -l "app.kubernetes.io/component=aggregator"
You can mount your own configuration files to the aggregators. For example, this is useful if you want to forward the logs to Elasticsearch or another service.
{{- else if and (not .Values.aggregator.enabled) (not .Values.forwarder.configMap) }}
Logs are captured on each node by the forwarder pods and sent to the standard output
Logs are captured on each node by the forwarder pods and sent to the standard output by default.
You can see all the logs by running this command:
kubectl logs -l "app.kubernetes.io/component=forwarder"
@@ -19,4 +26,5 @@
You can mount your own configuration files to the forwarders. For example, this is useful if you want to forward the logs to Elasticsearch or another service.
{{- end }}
{{- include "fluentd.validateValues" . }}
{{- include "fluentd.checkRollingTags" . -}}
bitnami/fluentd/templates/_helpers.tpl
+23
View File
@@ -114,6 +114,7 @@ Validate data
*/}}
{{- define "fluentd.validateValues" -}}
{{- $messages := list -}}
{{- $messages := append $messages (include "fluentd.validateValues.deployment" .) -}}
{{- $messages := append $messages (include "fluentd.validateValues.rbac" .) -}}
{{- $messages := without $messages "" -}}
{{- $message := join "\n" $messages -}}
@@ -122,6 +123,15 @@ Validate data
{{- end -}}
{{- end -}}
{{/* Validate values of Fluentd - forwarders and aggregators can't be disabled at the same time */}}
{{- define "fluentd.validateValues.deployment" -}}
{{- if and (not .Values.forwarder.enabled) (not .Values.aggregator.enabled) -}}
fluentd:
You have disabled both the forwarders and the aggregators.
Please enable at least one of them (--set forwarder.enabled=true) (--set aggregator.enabled=true)
{{- end -}}
{{- end -}}
{{/* Validate values of Fluentd - must create serviceAccount to create enable RBAC */}}
{{- define "fluentd.validateValues.rbac" -}}
{{- if and .Values.rbac.create (not .Values.serviceAccount.create) -}}
@@ -163,3 +173,16 @@ Get the certificates secret name.
{{- printf "%s-tls" (include "fluentd.fullname" . ) -}}
{{- end -}}
{{- end -}}
{{/*
Renders a value that contains template.
Usage:
{{ include "fluentd.tplValue" (dict "value" .Values.path.to.the.Value "context" $) }}
*/}}
{{- define "fluentd.tplValue" -}}
{{- if typeIs "string" .value }}
{{- tpl .value .context }}
{{- else }}
{{- tpl (.value | toYaml) .context }}
{{- end }}
{{- end -}}
bitnami/fluentd/templates/aggregator-configmap.yaml
+3 -1
View File
@@ -37,12 +37,14 @@ data:
@type null
</match>
# TCP input to receive logs from the forwarders
# TCP input to receive logs from
{{- if and .Values.aggregator.port }}
<source>
@type forward
bind 0.0.0.0
port {{ .Values.aggregator.port }}
</source>
{{- end }}
# HTTP input for the liveness and readiness probes
<source>
bitnami/fluentd/templates/aggregator-statefulset.yaml
+6 -4
View File
@@ -19,7 +19,7 @@ spec:
annotations:
checksum/config: {{ include (print $.Template.BasePath "/aggregator-configmap.yaml") . | sha256sum }}
{{- if .Values.aggregator.podAnnotations }}
{{- toYaml .Values.aggregator.podAnnotations | nindent 8 }}
{{- include "fluentd.tplValue" (dict "value" .Values.aggregator.podAnnotations "context" $) | nindent 8 }}
{{- end }}
spec:
{{- include "fluentd.imagePullSecrets" . | nindent 6 }}
@@ -41,12 +41,14 @@ spec:
{{- toYaml .Values.aggregator.extraEnv | nindent 12 }}
{{- end }}
ports:
- name: http
containerPort: 9880
protocol: TCP
{{- if .Values.aggregator.port }}
- name: tcp
containerPort: {{ .Values.aggregator.port }}
protocol: TCP
{{- end }}
{{- if .Values.aggregator.containerPorts }}
{{- toYaml .Values.aggregator.containerPorts | nindent 8 }}
{{- end }}
{{- if .Values.metrics.enabled }}
- name: metrics
containerPort: {{ .Values.metrics.service.port }}
bitnami/fluentd/templates/aggregator-svc.yaml
+32
View File
@@ -0,0 +1,32 @@
{{- if .Values.aggregator.service.ports }}
apiVersion: v1
kind: Service
metadata:
name: {{ include "fluentd.fullname" . }}-aggregator
labels: {{- include "fluentd.labels" . | nindent 4 }}
app.kubernetes.io/component: aggregator
{{- if .Values.aggregator.service.annotations }}
annotations: {{- include "fluentd.tplValue" (dict "value" .Values.aggregator.service.annotations "context" $) | nindent 4 }}
{{- end }}
spec:
type: {{ .Values.aggregator.service.type }}
{{- if and .Values.aggregator.service.loadBalancerIP (eq .Values.aggregator.service.type "LoadBalancer") }}
loadBalancerIP: {{ .Values.aggregator.service.loadBalancerIP }}
{{- end }}
{{- if and .Values.aggregator.service.loadBalancerSourceRanges (eq .Values.aggregator.service.type "LoadBalancer") }}
loadBalancerSourceRanges:
{{- with .Values.aggregator.service.loadBalancerSourceRanges }}
{{ toYaml . | nindent 4 }}
{{- end }}
{{- end }}
{{- if and (eq .Values.aggregator.service.type "ClusterIP") .Values.aggregator.service.clusterIP }}
clusterIP: {{ .Values.aggregator.service.clusterIP }}
{{- end }}
ports:
{{- range $key, $value := .Values.aggregator.service.ports }}
- name: {{ $key }}
{{ toYaml $value | nindent 6 }}
{{- end }}
selector: {{ include "fluentd.matchLabels" . | nindent 4 }}
app.kubernetes.io/component: aggregator
{{- end }}
bitnami/fluentd/templates/forwarder-configmap.yaml
+1 -1
View File
@@ -1,4 +1,4 @@
{{- if not .Values.forwarder.configMap -}}
{{- if and .Values.forwarder.enabled (not .Values.forwarder.configMap) -}}
apiVersion: v1
kind: ConfigMap
metadata:
bitnami/fluentd/templates/forwarder-daemonset.yaml
+6 -4
View File
@@ -1,3 +1,4 @@
{{- if .Values.forwarder.enabled }}
apiVersion: apps/v1
kind: DaemonSet
metadata:
@@ -16,7 +17,7 @@ spec:
annotations:
checksum/config: {{ include (print $.Template.BasePath "/forwarder-configmap.yaml") . | sha256sum }}
{{- if .Values.forwarder.podAnnotations }}
{{- toYaml .Values.forwarder.podAnnotations | nindent 8 }}
{{- include "fluentd.tplValue" (dict "value" .Values.forwarder.podAnnotations "context" $) | nindent 8 }}
{{- end }}
spec:
{{- include "fluentd.imagePullSecrets" . | nindent 6 }}
@@ -39,9 +40,9 @@ spec:
{{- toYaml .Values.forwarder.extraEnv | nindent 8 }}
{{- end }}
ports:
- name: http
containerPort: 9880
protocol: TCP
{{- if .Values.forwarder.containerPorts }}
{{- toYaml .Values.forwarder.containerPorts | nindent 8 }}
{{- end }}
{{- if .Values.metrics.enabled }}
- name: metrics
containerPort: {{ .Values.metrics.service.port }}
@@ -114,3 +115,4 @@ spec:
{{- with .Values.forwarder.tolerations }}
tolerations: {{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
bitnami/fluentd/templates/forwarder-svc.yaml
+32
View File
@@ -0,0 +1,32 @@
{{- if and .Values.forwarder.enabled .Values.forwarder.service.ports }}
apiVersion: v1
kind: Service
metadata:
name: {{ include "fluentd.fullname" . }}-forwarder
labels: {{- include "fluentd.labels" . | nindent 4 }}
app.kubernetes.io/component: forwarder
{{- if .Values.forwarder.service.annotations }}
annotations: {{- include "fluentd.tplValue" (dict "value" .Values.forwarder.service.annotations "context" $) | nindent 4 }}
{{- end }}
spec:
type: {{ .Values.forwarder.service.type }}
{{- if and .Values.forwarder.service.loadBalancerIP (eq .Values.forwarder.service.type "LoadBalancer") }}
loadBalancerIP: {{ .Values.forwarder.service.loadBalancerIP }}
{{- end }}
{{- if and .Values.forwarder.service.loadBalancerSourceRanges (eq .Values.forwarder.service.type "LoadBalancer") }}
loadBalancerSourceRanges:
{{- with .Values.forwarder.service.loadBalancerSourceRanges }}
{{ toYaml . | nindent 4 }}
{{- end }}
{{- end }}
{{- if and (eq .Values.forwarder.service.type "ClusterIP") .Values.forwarder.service.clusterIP }}
clusterIP: {{ .Values.forwarder.service.clusterIP }}
{{- end }}
ports:
{{- range $key, $value := .Values.forwarder.service.ports }}
- name: {{ $key }}
{{ toYaml $value | nindent 6 }}
{{- end }}
selector: {{ include "fluentd.matchLabels" . | nindent 4 }}
app.kubernetes.io/component: forwarder
{{- end }}
bitnami/fluentd/templates/svc.yaml → bitnami/fluentd/templates/metrics-svc.yaml
+2 -2
View File
@@ -2,9 +2,9 @@
apiVersion: v1
kind: Service
metadata:
name: {{ include "fluentd.fullname" . }}
name: {{ include "fluentd.fullname" . }}-metrics
labels: {{- include "fluentd.labels" . | nindent 4 }}
annotations: {{- tpl (toYaml .Values.metrics.service.annotations) $ | nindent 4 }}
annotations: {{- include "fluentd.tplValue" (dict "value" .Values.metrics.service.annotations "context" $) | nindent 4 }}
spec:
type: {{ .Values.metrics.service.type }}
{{- if and (eq .Values.metrics.service.type "LoadBalancer") .Values.metrics.service.loadBalancerIP }}
bitnami/fluentd/templates/svc-headless.yaml
+5 -4
View File
@@ -1,4 +1,4 @@
{{- if .Values.aggregator.enabled -}}
{{- if and .Values.aggregator.enabled .Values.aggregator.service.ports -}}
apiVersion: v1
kind: Service
metadata:
@@ -9,9 +9,10 @@ spec:
type: ClusterIP
clusterIP: None
ports:
- name: tcp
port: {{ .Values.aggregator.port }}
targetPort: tcp
{{- range $key, $value := .Values.aggregator.service.ports }}
- name: {{ $key }}
{{ toYaml $value | nindent 6 }}
{{- end }}
selector: {{- include "fluentd.matchLabels" . | nindent 4 }}
app.kubernetes.io/component: aggregator
{{- end -}}
bitnami/fluentd/values-production.yaml
+113 -2
View File
@@ -13,7 +13,7 @@ global: {}
image:
registry: docker.io
repository: bitnami/fluentd
tag: 1.7.4-debian-9-r0
tag: 1.7.4-debian-9-r9
## Specify a imagePullPolicy
## Defaults to 'Always' if image tag is 'latest', else set to 'IfNotPresent'
## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images
@@ -38,6 +38,7 @@ image:
clusterDomain: cluster.local
forwarder:
enabled: true
## Name of the config file that will be used by Fluentd at launch
## Fluentd will look for it under the /opt/bitnami/fluentd/conf directory
##
@@ -59,6 +60,68 @@ forwarder:
##
extraEnv: {}
containerPorts:
- name: http
containerPort: 9880
protocol: TCP
# - name: syslog-tcp
# containerPort: 5140
# protocol: TCP
# - name: syslog-udp
# containerPort: 5140
# protocol: UDP
# - name: tcp
# containerPort: 24224
# protocol: TCP
## Service parameters
##
service:
## Service type
##
type: ClusterIP
## Service ports
##
ports:
# http:
# port: 9880
# targetPort: http
# protocol: TCP
# syslog-udp:
# port: 5140
# targetPort: syslog-udp
# protocol: UDP
# nodePort: 31514
# syslog-tcp:
# port: 5140
# targetPort: syslog-tcp
# protocol: TCP
# nodePort: 31514
# tcp:
# port: 24224
# targetPort: tcp
# protocol: TCP
## loadBalancerIP for the forwarders service (optional, cloud specific)
## ref: http://kubernetes.io/docs/user-guide/services/#type-loadbalancer
# loadBalancerIP:
## Load Balancer sources
## https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/#restrict-access-for-loadbalancer-service
##
# loadBalancerSourceRanges:
# - 10.10.10.0/24
## Set the Cluster IP to use
## ref: https://kubernetes.io/docs/concepts/services-networking/service/#choosing-your-own-ip-address
##
# clusterIP: None
## Provide any additional annotations which may be required
##
annotations: {}
## Configure extra options for liveness and readiness probes
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#configure-probes)
##
@@ -133,7 +196,8 @@ aggregator:
## If not specified, one will be created by default
# configMap:
## Port which the forwarders will connect to to send the logs
## Port the aggregator container will listen for logs. Leave it blank to ignore.
## You can specify other ports in the aggregator.containerPorts parameter
##
port: 24224
@@ -149,6 +213,53 @@ aggregator:
##
extraEnv: {}
containerPorts:
# - name: my-port
# containerPort: 24222
# protocol: TCP
- name: http
containerPort: 9880
protocol: TCP
## Service parameters
##
service:
## Service type
##
type: ClusterIP
## Service ports
##
ports:
tcp:
port: 24224
targetPort: tcp
protocol: TCP
# http:
# port: 9880
# targetPort: http
# protocol: TCP
## loadBalancerIP for the forwarders service (optional, cloud specific)
## ref: http://kubernetes.io/docs/user-guide/services/#type-loadbalancer
##
# loadBalancerIP:
## Load Balancer sources
## https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/#restrict-access-for-loadbalancer-service
##
# loadBalancerSourceRanges:
# - 10.10.10.0/24
## Set the Cluster IP to use
## ref: https://kubernetes.io/docs/concepts/services-networking/service/#choosing-your-own-ip-address
##
# clusterIP: None
## Provide any additional annotations which may be required
##
annotations: {}
## Configure extra options for liveness and readiness probes
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#configure-probes)
##
bitnami/fluentd/values.yaml
+113 -2
View File
@@ -13,7 +13,7 @@ global: {}
image:
registry: docker.io
repository: bitnami/fluentd
tag: 1.7.4-debian-9-r0
tag: 1.7.4-debian-9-r9
## Specify a imagePullPolicy
## Defaults to 'Always' if image tag is 'latest', else set to 'IfNotPresent'
## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images
@@ -38,6 +38,7 @@ image:
clusterDomain: cluster.local
forwarder:
enabled: true
## Name of the config file that will be used by Fluentd at launch
## Fluentd will look for it under the /opt/bitnami/fluentd/conf directory
##
@@ -59,6 +60,68 @@ forwarder:
##
extraEnv: {}
containerPorts:
- name: http
containerPort: 9880
protocol: TCP
# - name: syslog-tcp
# containerPort: 5140
# protocol: TCP
# - name: syslog-udp
# containerPort: 5140
# protocol: UDP
# - name: tcp
# containerPort: 24224
# protocol: TCP
## Service parameters
##
service:
## Service type
##
type: ClusterIP
## Service ports
##
ports:
# http:
# port: 9880
# targetPort: http
# protocol: TCP
# syslog-udp:
# port: 5140
# targetPort: syslog-udp
# protocol: UDP
# nodePort: 31514
# syslog-tcp:
# port: 5140
# targetPort: syslog-tcp
# protocol: TCP
# nodePort: 31514
# tcp:
# port: 24224
# targetPort: tcp
# protocol: TCP
## loadBalancerIP for the forwarders service (optional, cloud specific)
## ref: http://kubernetes.io/docs/user-guide/services/#type-loadbalancer
# loadBalancerIP:
## Load Balancer sources
## https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/#restrict-access-for-loadbalancer-service
##
# loadBalancerSourceRanges:
# - 10.10.10.0/24
## Set the Cluster IP to use
## ref: https://kubernetes.io/docs/concepts/services-networking/service/#choosing-your-own-ip-address
##
# clusterIP: None
## Provide any additional annotations which may be required
##
annotations: {}
## Configure extra options for liveness and readiness probes
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#configure-probes)
##
@@ -133,7 +196,8 @@ aggregator:
## If not specified, one will be created by default
# configMap:
## Port which the forwarders will connect to to send the logs
## Port the aggregator container will listen for logs. Leave it blank to ignore.
## You can specify other ports in the aggregator.containerPorts parameter
##
port: 24224
@@ -149,6 +213,53 @@ aggregator:
##
extraEnv: {}
containerPorts:
# - name: my-port
# containerPort: 24222
# protocol: TCP
- name: http
containerPort: 9880
protocol: TCP
## Service parameters
##
service:
## Service type
##
type: ClusterIP
## Service ports
##
ports:
tcp:
port: 24224
targetPort: tcp
protocol: TCP
# http:
# port: 9880
# targetPort: http
# protocol: TCP
## loadBalancerIP for the forwarders service (optional, cloud specific)
## ref: http://kubernetes.io/docs/user-guide/services/#type-loadbalancer
##
# loadBalancerIP:
## Load Balancer sources
## https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/#restrict-access-for-loadbalancer-service
##
# loadBalancerSourceRanges:
# - 10.10.10.0/24
## Set the Cluster IP to use
## ref: https://kubernetes.io/docs/concepts/services-networking/service/#choosing-your-own-ip-address
##
# clusterIP: None
## Provide any additional annotations which may be required
##
annotations: {}
## Configure extra options for liveness and readiness probes
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#configure-probes)
##