mirror of
https://github.com/wahyd4/charts.git
synced 2026-08-23 03:56:24 +10:00
[bitnami/kafka] Add paramters to configure SASL_SCRAM (#3270)
* [bitnami/kafka] Add paramters to configure SASL_SCRAM * Add feature to create multiple users * Use an array instead of string for users and passwords * Remove old users configuration * Fix password in metrics deployment * Add suggestions * Fix version
This commit is contained in:
Miguel Ángel Cabrera Miñagorri
@@ -1,6 +1,6 @@
|
||||
apiVersion: v1
|
||||
name: kafka
|
||||
version: 11.6.6
|
||||
version: 11.7.0
|
||||
appVersion: 2.5.0
|
||||
description: Apache Kafka is a distributed streaming platform.
|
||||
keywords:
|
||||
|
||||
+16
-9
@@ -109,16 +109,18 @@ The following tables lists the configurable parameters of the Kafka chart and th
|
||||
| `extraVolumeMounts` | Extra volumeMount(s) to add to Kafka containers | `[]` |
|
||||
| `auth.clientProtocol` | Authentication protocol for communications with clients. Allowed protocols: `plaintext`, `tls`, `mtls`, `sasl` and `sasl_tls` | `plaintext` |
|
||||
| `auth.interBrokerProtocol` | Authentication protocol for inter-broker communications. Allowed protocols: `plaintext`, `tls`, `mtls`, `sasl` and `sasl_tls` | `plaintext` |
|
||||
| `auth.saslMechanisms` | SASL mechanisms when either `auth.interBrokerProtocol` or `auth.clientProtocol` are `sasl`. Allowed types: `plain`, `scram-sha-256`, `scram-sha-512` | `plain,scram-sha-256,scram-sha-512` |
|
||||
| `auth.saslInterBrokerMechanism` | SASL mechanism to use as inter broker protocol, it must be included at `auth.saslMechanisms` | `plain` |
|
||||
| `auth.jksSecret` | Name of the existing secret containing the truststore and one keystore per Kafka broker you have in the cluster | `nil` |
|
||||
| `auth.jksPassword` | Password to access the JKS files when they are password-protected | `nil` |
|
||||
| `auth.tlsEndpointIdentificationAlgorithm` | The endpoint identification algorithm to validate server hostname using server certificate | `https` |
|
||||
| `auth.jaas.brokerUser` | Kafka client user for SASL authentication | `user` |
|
||||
| `auth.jaas.brokerPassword` | Kafka client password for SASL authentication | `nil` |
|
||||
| `auth.jaas.interBrokerUser` | Kafka inter broker communication user for SASL authentication | `admin` |
|
||||
| `auth.jaas.interBrokerPassword` | Kafka inter broker communication password for SASL authentication | `nil` |
|
||||
| `auth.jaas.zookeeperUser` | Kafka Zookeeper user for SASL authentication | `nil` |
|
||||
| `auth.jaas.zookeeperPassword` | Kafka Zookeeper password for SASL authentication | `nil` |
|
||||
| `auth.jaas.existingSecret` | Name of the existing secret containing credentials for brokerUser, interBrokerUser and zookeeperUser | `nil` |
|
||||
| `auth.jaas.clientUsers` | List of Kafka client users to be created, separated by commas. This values will override `auth.jaas.clientUser` | `[]` |
|
||||
| `auth.jaas.clientPasswords` | List of passwords for `auth.jaas.clientUsers`. It is mandatory to provide the passwords when using `auth.jaas.clientUsers` | `[]` |
|
||||
| `listeners` | The address(es) the socket server listens on. Auto-calculated it's set to an empty array | `[]` |
|
||||
| `advertisedListeners` | The address(es) (hostname:port) the broker will advertise to producers and consumers. Auto-calculated it's set to an empty array | `[]` |
|
||||
| `listenerSecurityProtocolMap` | The protocol->listener mapping. Auto-calculated it's set to nil | `nil` |
|
||||
@@ -144,8 +146,8 @@ The following tables lists the configurable parameters of the Kafka chart and th
|
||||
| `resources.requests` | The requested resources for Kafka containers | `{}` |
|
||||
| `livenessProbe` | Liveness probe configuration for Kafka | `Check values.yaml file` |
|
||||
| `readinessProbe` | Readiness probe configuration for Kafka | `Check values.yaml file` |
|
||||
| `customLivenessProbe` | Custom Liveness probe configuration for Kafka | `{}` |
|
||||
| `customReadinessProbe` | Custom Readiness probe configuration for Kafka | `{}` |
|
||||
| `customLivenessProbe` | Custom Liveness probe configuration for Kafka | `{}` |
|
||||
| `customReadinessProbe` | Custom Readiness probe configuration for Kafka | `{}` |
|
||||
| `pdb.create` | Enable/disable a Pod Disruption Budget creation | `false` |
|
||||
| `pdb.minAvailable` | Minimum number/percentage of pods that should remain scheduled | `nil` |
|
||||
| `pdb.maxUnavailable` | Maximum number/percentage of pods that may be made unavailable | `1` |
|
||||
@@ -414,7 +416,7 @@ You can configure different authentication protocols for each listener you confi
|
||||
|
||||
If you enabled SASL authentication on any listener, you can set the SASL credentials using the parameters below:
|
||||
|
||||
- `auth.jaas.clientUser`/`auth.jaas.clientPassword`: when enabling SASL authentication for communications with clients.
|
||||
- `auth.jaas.clientUsers`/`auth.jaas.clientPasswords`: when enabling SASL authentication for communications with clients.
|
||||
- `auth.jaas.interBrokerUser`/`auth.jaas.interBrokerPassword`: when enabling SASL authentication for inter-broker communications.
|
||||
- `auth.jaas.zookeeperUser`/`auth.jaas.zookeeperPassword`: In the case that the Zookeeper chart is deployed with SASL authentication enabled.
|
||||
|
||||
@@ -440,8 +442,8 @@ auth.clientProtocol=sasl
|
||||
auth.interBrokerProtocol=tls
|
||||
auth.certificatesSecret=kafka-jks
|
||||
auth.certificatesPassword=jksPassword
|
||||
auth.jaas.clientUser=brokerUser
|
||||
auth.jaas.clientPassword=brokerPassword
|
||||
auth.jaas.clientUsers[0]=brokerUser
|
||||
auth.jaas.clientPassword[0]=brokerPassword
|
||||
auth.jaas.zookeeperUser=zookeeperUser
|
||||
auth.jaas.zookeeperPassword=zookeeperPassword
|
||||
zookeeper.auth.enabled=true
|
||||
@@ -629,6 +631,13 @@ You can enable this initContainer by setting `volumePermissions.enabled` to `tru
|
||||
|
||||
## Upgrading
|
||||
|
||||
### To 11.7.0
|
||||
|
||||
The way to configure the users and passwords changed. Now it is allowed to create multiple users during the installation by providing the list of users and passwords.
|
||||
|
||||
- `auth.jaas.clientUser` (string) -> deprecated in favor of `auth.jaas.clientUsers` (array).
|
||||
- `auth.jaas.clientPassword` (string) -> deprecated in favor of `auth.jaas.clientPasswords` (array).
|
||||
|
||||
### To 11.0.0
|
||||
|
||||
The way to configure listeners and athentication on Kafka is totally refactored allowing users to configure different authentication protocols on different listeners. Please check the sections [Listeners Configuration](listeners-configuration) and [Listeners Configuration](enable-kafka-for-kafka-and-zookeeper) for more information.
|
||||
@@ -640,8 +649,6 @@ Backwards compatibility is not guaranteed you adapt your values.yaml to the new
|
||||
- `auth.certificatesSecret` -> renamed to `auth.jksSecret`.
|
||||
- `auth.certificatesPassword` -> renamed to `auth.jksPassword`.
|
||||
- `sslEndpointIdentificationAlgorithm` -> renamedo to `auth.tlsEndpointIdentificationAlgorithm`.
|
||||
- `auth.brokerUser` -> renamed to `auth.jaas.clientUser`
|
||||
- `auth.brokerPassword` -> renamed to `auth.jaas.clientPassword`
|
||||
- `auth.interBrokerUser` -> renamed to `auth.jaas.interBrokerUser`
|
||||
- `auth.interBrokerPassword` -> renamed to `auth.jaas.interBrokerPassword`
|
||||
- `auth.zookeeperUser` -> renamed to `auth.jaas.zookeeperUser`
|
||||
|
||||
@@ -73,15 +73,25 @@ You need to configure your Kafka client to access using SASL authentication. To
|
||||
- kafka_jaas.conf:
|
||||
|
||||
KafkaClient {
|
||||
{{- if .Values.auth.saslMechanisms | regexFind "scram" }}
|
||||
org.apache.kafka.common.security.scram.ScramLoginModule required
|
||||
{{- else }}
|
||||
org.apache.kafka.common.security.plain.PlainLoginModule required
|
||||
username="{{ .Values.auth.jaas.clientUser }}"
|
||||
password="$(kubectl get secret {{ $fullname }}-jaas -n {{ $releaseNamespace }} -o jsonpath='{.data.client-password}' | base64 --decode)";
|
||||
{{- end }}
|
||||
username="{{ index .Values.auth.jaas.clientUsers 0 }}"
|
||||
password="$(kubectl get secret {{ $fullname }}-jaas -n {{ $releaseNamespace }} -o jsonpath='{.data.client-passwords}' | base64 --decode | cut -d , -f 1)";
|
||||
};
|
||||
|
||||
- client.properties:
|
||||
|
||||
security.protocol={{ $clientProtocol }}
|
||||
{{- if .Values.auth.saslMechanisms | regexFind "scram-sha-256" }}
|
||||
sasl.mechanism=SCRAM-SHA-256
|
||||
{{- else if .Values.auth.saslMechanisms | regexFind "scram-sha-512" }}
|
||||
sasl.mechanism=SCRAM-SHA-512
|
||||
{{- else -}}
|
||||
sasl.mechanism=PLAIN
|
||||
{{- end }}
|
||||
{{- if eq .Values.auth.clientProtocol "sasl_tls" }}
|
||||
ssl.truststore.location=/tmp/kafka.truststore.jks
|
||||
{{- if .Values.auth.jksPassword }}
|
||||
|
||||
@@ -323,6 +323,26 @@ SASL_PLAINTEXT
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
||||
{{/*
|
||||
Return the SASL type
|
||||
Usage:
|
||||
{{ include "kafka.auth.saslMechanisms" ( dict "type" .Values.path.to.the.Value ) }}
|
||||
*/}}
|
||||
{{- define "kafka.auth.saslMechanisms" -}}
|
||||
{{- $mechanisms := list -}}
|
||||
{{- if .type | regexFind "plain" -}}
|
||||
{{- $mechanisms = append $mechanisms "PLAIN" -}}
|
||||
{{- end -}}
|
||||
{{- if .type | regexFind "scram-sha-256" -}}
|
||||
{{- $mechanisms = append $mechanisms "SCRAM-SHA-256" -}}
|
||||
{{- end -}}
|
||||
{{- if .type | regexFind "scram-sha-512" -}}
|
||||
{{- $mechanisms = append $mechanisms "SCRAM-SHA-512" -}}
|
||||
{{- end -}}
|
||||
{{- $mechanisms = join "," $mechanisms -}}
|
||||
{{- printf "%s" $mechanisms -}}
|
||||
{{- end -}}
|
||||
|
||||
{{/*
|
||||
Return the Kafka JAAS credentials secret
|
||||
*/}}
|
||||
@@ -468,6 +488,7 @@ Compile all warnings into a single message, and call fail.
|
||||
{{- $messages := append $messages (include "kafka.validateValues.externalAccessServiceType" .) -}}
|
||||
{{- $messages := append $messages (include "kafka.validateValues.externalAccessAutoDiscoveryRBAC" .) -}}
|
||||
{{- $messages := append $messages (include "kafka.validateValues.jksSecret" .) -}}
|
||||
{{- $messages := append $messages (include "kafka.validateValues.saslMechanisms" .) -}}
|
||||
{{- $messages := without $messages "" -}}
|
||||
{{- $message := join "\n" $messages -}}
|
||||
|
||||
@@ -521,3 +542,15 @@ kafka: auth.jksSecret
|
||||
A secret containing the Kafka JKS files is required when TLS encryption in enabled
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
||||
{{/* Validate values of Kafka - SASL mechanisms must be provided when using SASL */}}
|
||||
{{- define "kafka.validateValues.saslMechanisms" -}}
|
||||
{{- if and (or (.Values.auth.clientProtocol | regexFind "sasl") (.Values.auth.interBrokerProtocol | regexFind "sasl") .Values.auth.jaas.zookeeperUser) (not .Values.auth.saslMechanisms) }}
|
||||
kafka: auth.saslMechanisms
|
||||
The SASL mechanisms are required when either auth.clientProtocol or auth.interBrokerProtocol use SASL or Zookeeper user is provided.
|
||||
{{- end }}
|
||||
{{- if not (contains .Values.auth.saslInterBrokerMechanism .Values.auth.saslMechanisms) }}
|
||||
kafka: auth.saslMechanisms
|
||||
auth.saslInterBrokerMechanism must be provided and it should be one of the specified mechanisms at auth.saslMechanisms
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
||||
@@ -13,7 +13,15 @@ metadata:
|
||||
type: Opaque
|
||||
data:
|
||||
{{- if (include "kafka.client.saslAuthentication" .) }}
|
||||
client-password: {{ ternary (randAlphaNum 10) .Values.auth.jaas.clientPassword (empty .Values.auth.jaas.clientPassword) | b64enc | quote }}
|
||||
{{- if .Values.auth.jaas.clientPasswords }}
|
||||
client-passwords: {{ join "," .Values.auth.jaas.clientPasswords | b64enc | quote }}
|
||||
{{- else }}
|
||||
{{- $passwords := list }}
|
||||
{{- range .Values.auth.jaas.clientUsers }}
|
||||
{{- $passwords = append $passwords (randAlphaNum 10) }}
|
||||
{{- end }}
|
||||
client-passwords: {{ join "," $passwords | b64enc | quote }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- if .Values.auth.jaas.zookeeperUser }}
|
||||
zookeeper-password: {{ ternary (randAlphaNum 10) .Values.auth.jaas.zookeeperPassword (empty .Values.auth.jaas.zookeeperPassword) | b64enc | quote }}
|
||||
|
||||
@@ -35,14 +35,15 @@ spec:
|
||||
- /bin/bash
|
||||
- -ec
|
||||
- |
|
||||
read -r -a sasl_passwords <<< "$(tr ',;' ' ' <<< "${SASL_USER_PASSWORD}")"
|
||||
kafka_exporter \
|
||||
{{- range $i, $e := until $replicaCount }}
|
||||
--kafka.server={{ $fullname }}-{{ $i }}.{{ $fullname }}-headless.{{ $releaseNamespace }}.svc.{{ $clusterDomain }}:{{ $servicePort }} \
|
||||
{{- end }}
|
||||
{{- if (include "kafka.client.saslAuthentication" .) }}
|
||||
--sasl.enabled \
|
||||
--sasl.username={{ .Values.auth.jaas.clientUser | quote }} \
|
||||
--sasl.password="$SASL_USER_PASSWORD" \
|
||||
--sasl.username="$SASL_USERNAME" \
|
||||
--sasl.password="${sasl_passwords[0]}" \
|
||||
{{- end }}
|
||||
{{- if (include "kafka.tlsEncryption" .) }}
|
||||
--tls.enabled \
|
||||
@@ -58,11 +59,13 @@ spec:
|
||||
--web.listen-address=:9308
|
||||
{{- if (include "kafka.client.saslAuthentication" .) }}
|
||||
env:
|
||||
- name: SASL_USERNAME
|
||||
value: {{ index .Values.auth.jaas.clientUsers 0 | quote }}
|
||||
- name: SASL_USER_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: {{ include "kafka.jaasSecretName" . }}
|
||||
key: client-password
|
||||
key: client-passwords
|
||||
{{- end }}
|
||||
ports:
|
||||
- name: metrics
|
||||
|
||||
@@ -161,6 +161,12 @@ spec:
|
||||
{{- else }}
|
||||
value: "INTERNAL:{{ $interBrokerProtocol }},CLIENT:{{ $clientProtocol }}"
|
||||
{{- end }}
|
||||
{{- if or ($clientProtocol | regexFind "SASL") ($interBrokerProtocol | regexFind "SASL") .Values.auth.jaas.zookeeperUser }}
|
||||
- name: KAFKA_CFG_SASL_ENABLED_MECHANISMS
|
||||
value: {{ include "kafka.auth.saslMechanisms" ( dict "type" .Values.auth.saslMechanisms ) }}
|
||||
- name: KAFKA_CFG_SASL_MECHANISM_INTER_BROKER_PROTOCOL
|
||||
value: {{ upper .Values.auth.saslInterBrokerMechanism | quote }}
|
||||
{{- end }}
|
||||
- name: KAFKA_CFG_LISTENERS
|
||||
{{- if .Values.listeners }}
|
||||
value: {{ .Values.listeners }}
|
||||
@@ -188,13 +194,13 @@ spec:
|
||||
- name: KAFKA_OPTS
|
||||
value: "-Djava.security.auth.login.config=/opt/bitnami/kafka/conf/kafka_jaas.conf"
|
||||
{{- if (include "kafka.client.saslAuthentication" .) }}
|
||||
- name: KAFKA_CLIENT_USER
|
||||
value: {{ .Values.auth.jaas.clientUser | quote }}
|
||||
- name: KAFKA_CLIENT_PASSWORD
|
||||
- name: KAFKA_CLIENT_USERS
|
||||
value: {{ join "," .Values.auth.jaas.clientUsers | quote }}
|
||||
- name: KAFKA_CLIENT_PASSWORDS
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: {{ include "kafka.jaasSecretName" . }}
|
||||
key: client-password
|
||||
key: client-passwords
|
||||
{{- end }}
|
||||
{{- if .Values.auth.jaas.zookeeperUser }}
|
||||
- name: KAFKA_ZOOKEEPER_PROTOCOL
|
||||
|
||||
@@ -244,6 +244,13 @@ auth:
|
||||
clientProtocol: sasl
|
||||
interBrokerProtocol: sasl
|
||||
|
||||
## Allowed SASL mechanisms when clientProtocol or interBrokerProtocol are using either sasl or sasl_tls
|
||||
##
|
||||
saslMechanisms: plain,scram-sha-256,scram-sha-512
|
||||
## SASL mechanism for inter broker communication
|
||||
##
|
||||
saslInterBrokerMechanism: plain
|
||||
|
||||
## Name of the existing secret containing the truststore and
|
||||
## one keystore per Kafka broker you have in the Kafka cluster.
|
||||
## MANDATORY when 'tls', 'mtls', or 'sasl_tls' authentication protocols are used.
|
||||
@@ -271,13 +278,22 @@ auth:
|
||||
## MANDATORY when method is 'sasl', or 'sasl_tls'
|
||||
##
|
||||
jaas:
|
||||
## Kafka client user
|
||||
## Kafka client user list
|
||||
##
|
||||
clientUser: user
|
||||
## clientUsers:
|
||||
## - user1
|
||||
## - user2
|
||||
##
|
||||
clientUsers:
|
||||
- user
|
||||
|
||||
## Kafka client password
|
||||
## Kafka client passwords
|
||||
##
|
||||
clientPassword: ""
|
||||
## clientPasswords:
|
||||
## - password1
|
||||
## - password2
|
||||
##
|
||||
clientPasswords: []
|
||||
|
||||
## Kafka inter broker communication user
|
||||
##
|
||||
@@ -295,9 +311,9 @@ auth:
|
||||
##
|
||||
zookeeperPassword: zookeeperPassword
|
||||
|
||||
## Name of the existing secret containing credentials for clientUser, interBrokerUser and zookeeperUser.
|
||||
## Name of the existing secret containing credentials for clientUsers, interBrokerUser and zookeeperUser.
|
||||
## Create this secret running the command below where SECRET_NAME is the name of the secret you want to create:
|
||||
## kubectl create secret generic SECRET_NAME --from-literal=client-password=CLIENT_PASSWORD --from-literal=inter-broker-password=INTER_BROKER_PASSWORD --from-literal=zookeeper-password=ZOOKEEPER_PASSWORD
|
||||
## kubectl create secret generic SECRET_NAME --from-literal=client-password=CLIENT_PASSWORD1,CLIENT_PASSWORD2 --from-literal=inter-broker-password=INTER_BROKER_PASSWORD --from-literal=zookeeper-password=ZOOKEEPER_PASSWORD
|
||||
##
|
||||
# existingSecret:
|
||||
|
||||
|
||||
@@ -244,6 +244,13 @@ auth:
|
||||
clientProtocol: plaintext
|
||||
interBrokerProtocol: plaintext
|
||||
|
||||
## Allowed SASL mechanisms when clientProtocol or interBrokerProtocol are using either sasl or sasl_tls
|
||||
##
|
||||
saslMechanisms: plain,scram-sha-256,scram-sha-512
|
||||
## SASL mechanism for inter broker communication
|
||||
##
|
||||
saslInterBrokerMechanism: plain
|
||||
|
||||
## Name of the existing secret containing the truststore and
|
||||
## one keystore per Kafka broker you have in the Kafka cluster.
|
||||
## MANDATORY when 'tls', 'mtls', or 'sasl_tls' authentication protocols are used.
|
||||
@@ -271,13 +278,22 @@ auth:
|
||||
## MANDATORY when method is 'sasl', or 'sasl_tls'
|
||||
##
|
||||
jaas:
|
||||
## Kafka client user
|
||||
## Kafka client user list
|
||||
##
|
||||
clientUser: user
|
||||
## clientUsers:
|
||||
## - user1
|
||||
## - user2
|
||||
##
|
||||
clientUsers:
|
||||
- user
|
||||
|
||||
## Kafka client password
|
||||
## Kafka client passwords. This is mandatory if more than one user is specified in clientUsers.
|
||||
##
|
||||
clientPassword: ""
|
||||
## clientPasswords:
|
||||
## - password1
|
||||
## - password2"
|
||||
##
|
||||
clientPasswords: []
|
||||
|
||||
## Kafka inter broker communication user
|
||||
##
|
||||
@@ -295,9 +311,9 @@ auth:
|
||||
##
|
||||
# zookeeperPassword:
|
||||
|
||||
## Name of the existing secret containing credentials for clientUser, interBrokerUser and zookeeperUser.
|
||||
## Name of the existing secret containing credentials for clientUsers, interBrokerUser and zookeeperUser.
|
||||
## Create this secret running the command below where SECRET_NAME is the name of the secret you want to create:
|
||||
## kubectl create secret generic SECRET_NAME --from-literal=client-password=CLIENT_PASSWORD --from-literal=inter-broker-password=INTER_BROKER_PASSWORD --from-literal=zookeeper-password=ZOOKEEPER_PASSWORD
|
||||
## kubectl create secret generic SECRET_NAME --from-literal=client-passwords=CLIENT_PASSWORD1,CLIENT_PASSWORD2 --from-literal=inter-broker-password=INTER_BROKER_PASSWORD --from-literal=zookeeper-password=ZOOKEEPER_PASSWORD
|
||||
##
|
||||
# existingSecret:
|
||||
|
||||
|
||||
Reference in New Issue
Block a user