diff --git a/bitnami/kafka/Chart.yaml b/bitnami/kafka/Chart.yaml index 4f6531362..54dead7b9 100644 --- a/bitnami/kafka/Chart.yaml +++ b/bitnami/kafka/Chart.yaml @@ -1,6 +1,6 @@ apiVersion: v1 name: kafka -version: 11.6.6 +version: 11.7.0 appVersion: 2.5.0 description: Apache Kafka is a distributed streaming platform. keywords: diff --git a/bitnami/kafka/README.md b/bitnami/kafka/README.md index f8cbba15f..c65d92806 100644 --- a/bitnami/kafka/README.md +++ b/bitnami/kafka/README.md @@ -109,16 +109,18 @@ The following tables lists the configurable parameters of the Kafka chart and th | `extraVolumeMounts` | Extra volumeMount(s) to add to Kafka containers | `[]` | | `auth.clientProtocol` | Authentication protocol for communications with clients. Allowed protocols: `plaintext`, `tls`, `mtls`, `sasl` and `sasl_tls` | `plaintext` | | `auth.interBrokerProtocol` | Authentication protocol for inter-broker communications. Allowed protocols: `plaintext`, `tls`, `mtls`, `sasl` and `sasl_tls` | `plaintext` | +| `auth.saslMechanisms` | SASL mechanisms when either `auth.interBrokerProtocol` or `auth.clientProtocol` are `sasl`. Allowed types: `plain`, `scram-sha-256`, `scram-sha-512` | `plain,scram-sha-256,scram-sha-512` | +| `auth.saslInterBrokerMechanism` | SASL mechanism to use as inter broker protocol, it must be included at `auth.saslMechanisms` | `plain` | | `auth.jksSecret` | Name of the existing secret containing the truststore and one keystore per Kafka broker you have in the cluster | `nil` | | `auth.jksPassword` | Password to access the JKS files when they are password-protected | `nil` | | `auth.tlsEndpointIdentificationAlgorithm` | The endpoint identification algorithm to validate server hostname using server certificate | `https` | -| `auth.jaas.brokerUser` | Kafka client user for SASL authentication | `user` | -| `auth.jaas.brokerPassword` | Kafka client password for SASL authentication | `nil` | | `auth.jaas.interBrokerUser` | Kafka inter broker communication user for SASL authentication | `admin` | | `auth.jaas.interBrokerPassword` | Kafka inter broker communication password for SASL authentication | `nil` | | `auth.jaas.zookeeperUser` | Kafka Zookeeper user for SASL authentication | `nil` | | `auth.jaas.zookeeperPassword` | Kafka Zookeeper password for SASL authentication | `nil` | | `auth.jaas.existingSecret` | Name of the existing secret containing credentials for brokerUser, interBrokerUser and zookeeperUser | `nil` | +| `auth.jaas.clientUsers` | List of Kafka client users to be created, separated by commas. This values will override `auth.jaas.clientUser` | `[]` | +| `auth.jaas.clientPasswords` | List of passwords for `auth.jaas.clientUsers`. It is mandatory to provide the passwords when using `auth.jaas.clientUsers` | `[]` | | `listeners` | The address(es) the socket server listens on. Auto-calculated it's set to an empty array | `[]` | | `advertisedListeners` | The address(es) (hostname:port) the broker will advertise to producers and consumers. Auto-calculated it's set to an empty array | `[]` | | `listenerSecurityProtocolMap` | The protocol->listener mapping. Auto-calculated it's set to nil | `nil` | @@ -144,8 +146,8 @@ The following tables lists the configurable parameters of the Kafka chart and th | `resources.requests` | The requested resources for Kafka containers | `{}` | | `livenessProbe` | Liveness probe configuration for Kafka | `Check values.yaml file` | | `readinessProbe` | Readiness probe configuration for Kafka | `Check values.yaml file` | -| `customLivenessProbe` | Custom Liveness probe configuration for Kafka | `{}` | -| `customReadinessProbe` | Custom Readiness probe configuration for Kafka | `{}` | +| `customLivenessProbe` | Custom Liveness probe configuration for Kafka | `{}` | +| `customReadinessProbe` | Custom Readiness probe configuration for Kafka | `{}` | | `pdb.create` | Enable/disable a Pod Disruption Budget creation | `false` | | `pdb.minAvailable` | Minimum number/percentage of pods that should remain scheduled | `nil` | | `pdb.maxUnavailable` | Maximum number/percentage of pods that may be made unavailable | `1` | @@ -414,7 +416,7 @@ You can configure different authentication protocols for each listener you confi If you enabled SASL authentication on any listener, you can set the SASL credentials using the parameters below: -- `auth.jaas.clientUser`/`auth.jaas.clientPassword`: when enabling SASL authentication for communications with clients. +- `auth.jaas.clientUsers`/`auth.jaas.clientPasswords`: when enabling SASL authentication for communications with clients. - `auth.jaas.interBrokerUser`/`auth.jaas.interBrokerPassword`: when enabling SASL authentication for inter-broker communications. - `auth.jaas.zookeeperUser`/`auth.jaas.zookeeperPassword`: In the case that the Zookeeper chart is deployed with SASL authentication enabled. @@ -440,8 +442,8 @@ auth.clientProtocol=sasl auth.interBrokerProtocol=tls auth.certificatesSecret=kafka-jks auth.certificatesPassword=jksPassword -auth.jaas.clientUser=brokerUser -auth.jaas.clientPassword=brokerPassword +auth.jaas.clientUsers[0]=brokerUser +auth.jaas.clientPassword[0]=brokerPassword auth.jaas.zookeeperUser=zookeeperUser auth.jaas.zookeeperPassword=zookeeperPassword zookeeper.auth.enabled=true @@ -629,6 +631,13 @@ You can enable this initContainer by setting `volumePermissions.enabled` to `tru ## Upgrading +### To 11.7.0 + +The way to configure the users and passwords changed. Now it is allowed to create multiple users during the installation by providing the list of users and passwords. + +- `auth.jaas.clientUser` (string) -> deprecated in favor of `auth.jaas.clientUsers` (array). +- `auth.jaas.clientPassword` (string) -> deprecated in favor of `auth.jaas.clientPasswords` (array). + ### To 11.0.0 The way to configure listeners and athentication on Kafka is totally refactored allowing users to configure different authentication protocols on different listeners. Please check the sections [Listeners Configuration](listeners-configuration) and [Listeners Configuration](enable-kafka-for-kafka-and-zookeeper) for more information. @@ -640,8 +649,6 @@ Backwards compatibility is not guaranteed you adapt your values.yaml to the new - `auth.certificatesSecret` -> renamed to `auth.jksSecret`. - `auth.certificatesPassword` -> renamed to `auth.jksPassword`. - `sslEndpointIdentificationAlgorithm` -> renamedo to `auth.tlsEndpointIdentificationAlgorithm`. -- `auth.brokerUser` -> renamed to `auth.jaas.clientUser` -- `auth.brokerPassword` -> renamed to `auth.jaas.clientPassword` - `auth.interBrokerUser` -> renamed to `auth.jaas.interBrokerUser` - `auth.interBrokerPassword` -> renamed to `auth.jaas.interBrokerPassword` - `auth.zookeeperUser` -> renamed to `auth.jaas.zookeeperUser` diff --git a/bitnami/kafka/templates/NOTES.txt b/bitnami/kafka/templates/NOTES.txt index c0d36a1c2..f6deb1223 100644 --- a/bitnami/kafka/templates/NOTES.txt +++ b/bitnami/kafka/templates/NOTES.txt @@ -73,15 +73,25 @@ You need to configure your Kafka client to access using SASL authentication. To - kafka_jaas.conf: KafkaClient { + {{- if .Values.auth.saslMechanisms | regexFind "scram" }} + org.apache.kafka.common.security.scram.ScramLoginModule required + {{- else }} org.apache.kafka.common.security.plain.PlainLoginModule required - username="{{ .Values.auth.jaas.clientUser }}" - password="$(kubectl get secret {{ $fullname }}-jaas -n {{ $releaseNamespace }} -o jsonpath='{.data.client-password}' | base64 --decode)"; + {{- end }} + username="{{ index .Values.auth.jaas.clientUsers 0 }}" + password="$(kubectl get secret {{ $fullname }}-jaas -n {{ $releaseNamespace }} -o jsonpath='{.data.client-passwords}' | base64 --decode | cut -d , -f 1)"; }; - client.properties: security.protocol={{ $clientProtocol }} + {{- if .Values.auth.saslMechanisms | regexFind "scram-sha-256" }} + sasl.mechanism=SCRAM-SHA-256 + {{- else if .Values.auth.saslMechanisms | regexFind "scram-sha-512" }} + sasl.mechanism=SCRAM-SHA-512 + {{- else -}} sasl.mechanism=PLAIN + {{- end }} {{- if eq .Values.auth.clientProtocol "sasl_tls" }} ssl.truststore.location=/tmp/kafka.truststore.jks {{- if .Values.auth.jksPassword }} diff --git a/bitnami/kafka/templates/_helpers.tpl b/bitnami/kafka/templates/_helpers.tpl index 66e6b32c3..cbe8db1c9 100644 --- a/bitnami/kafka/templates/_helpers.tpl +++ b/bitnami/kafka/templates/_helpers.tpl @@ -323,6 +323,26 @@ SASL_PLAINTEXT {{- end -}} {{- end -}} +{{/* +Return the SASL type +Usage: +{{ include "kafka.auth.saslMechanisms" ( dict "type" .Values.path.to.the.Value ) }} +*/}} +{{- define "kafka.auth.saslMechanisms" -}} +{{- $mechanisms := list -}} +{{- if .type | regexFind "plain" -}} +{{- $mechanisms = append $mechanisms "PLAIN" -}} +{{- end -}} +{{- if .type | regexFind "scram-sha-256" -}} +{{- $mechanisms = append $mechanisms "SCRAM-SHA-256" -}} +{{- end -}} +{{- if .type | regexFind "scram-sha-512" -}} +{{- $mechanisms = append $mechanisms "SCRAM-SHA-512" -}} +{{- end -}} +{{- $mechanisms = join "," $mechanisms -}} +{{- printf "%s" $mechanisms -}} +{{- end -}} + {{/* Return the Kafka JAAS credentials secret */}} @@ -468,6 +488,7 @@ Compile all warnings into a single message, and call fail. {{- $messages := append $messages (include "kafka.validateValues.externalAccessServiceType" .) -}} {{- $messages := append $messages (include "kafka.validateValues.externalAccessAutoDiscoveryRBAC" .) -}} {{- $messages := append $messages (include "kafka.validateValues.jksSecret" .) -}} +{{- $messages := append $messages (include "kafka.validateValues.saslMechanisms" .) -}} {{- $messages := without $messages "" -}} {{- $message := join "\n" $messages -}} @@ -521,3 +542,15 @@ kafka: auth.jksSecret A secret containing the Kafka JKS files is required when TLS encryption in enabled {{- end -}} {{- end -}} + +{{/* Validate values of Kafka - SASL mechanisms must be provided when using SASL */}} +{{- define "kafka.validateValues.saslMechanisms" -}} +{{- if and (or (.Values.auth.clientProtocol | regexFind "sasl") (.Values.auth.interBrokerProtocol | regexFind "sasl") .Values.auth.jaas.zookeeperUser) (not .Values.auth.saslMechanisms) }} +kafka: auth.saslMechanisms + The SASL mechanisms are required when either auth.clientProtocol or auth.interBrokerProtocol use SASL or Zookeeper user is provided. +{{- end }} +{{- if not (contains .Values.auth.saslInterBrokerMechanism .Values.auth.saslMechanisms) }} +kafka: auth.saslMechanisms + auth.saslInterBrokerMechanism must be provided and it should be one of the specified mechanisms at auth.saslMechanisms +{{- end -}} +{{- end -}} diff --git a/bitnami/kafka/templates/jaas-secret.yaml b/bitnami/kafka/templates/jaas-secret.yaml index fa52a7beb..0de8ce85d 100644 --- a/bitnami/kafka/templates/jaas-secret.yaml +++ b/bitnami/kafka/templates/jaas-secret.yaml @@ -13,7 +13,15 @@ metadata: type: Opaque data: {{- if (include "kafka.client.saslAuthentication" .) }} - client-password: {{ ternary (randAlphaNum 10) .Values.auth.jaas.clientPassword (empty .Values.auth.jaas.clientPassword) | b64enc | quote }} + {{- if .Values.auth.jaas.clientPasswords }} + client-passwords: {{ join "," .Values.auth.jaas.clientPasswords | b64enc | quote }} + {{- else }} + {{- $passwords := list }} + {{- range .Values.auth.jaas.clientUsers }} + {{- $passwords = append $passwords (randAlphaNum 10) }} + {{- end }} + client-passwords: {{ join "," $passwords | b64enc | quote }} + {{- end }} {{- end }} {{- if .Values.auth.jaas.zookeeperUser }} zookeeper-password: {{ ternary (randAlphaNum 10) .Values.auth.jaas.zookeeperPassword (empty .Values.auth.jaas.zookeeperPassword) | b64enc | quote }} diff --git a/bitnami/kafka/templates/kafka-metrics-deployment.yaml b/bitnami/kafka/templates/kafka-metrics-deployment.yaml index ba45775ce..f55f0457d 100644 --- a/bitnami/kafka/templates/kafka-metrics-deployment.yaml +++ b/bitnami/kafka/templates/kafka-metrics-deployment.yaml @@ -35,14 +35,15 @@ spec: - /bin/bash - -ec - | + read -r -a sasl_passwords <<< "$(tr ',;' ' ' <<< "${SASL_USER_PASSWORD}")" kafka_exporter \ {{- range $i, $e := until $replicaCount }} --kafka.server={{ $fullname }}-{{ $i }}.{{ $fullname }}-headless.{{ $releaseNamespace }}.svc.{{ $clusterDomain }}:{{ $servicePort }} \ {{- end }} {{- if (include "kafka.client.saslAuthentication" .) }} --sasl.enabled \ - --sasl.username={{ .Values.auth.jaas.clientUser | quote }} \ - --sasl.password="$SASL_USER_PASSWORD" \ + --sasl.username="$SASL_USERNAME" \ + --sasl.password="${sasl_passwords[0]}" \ {{- end }} {{- if (include "kafka.tlsEncryption" .) }} --tls.enabled \ @@ -58,11 +59,13 @@ spec: --web.listen-address=:9308 {{- if (include "kafka.client.saslAuthentication" .) }} env: + - name: SASL_USERNAME + value: {{ index .Values.auth.jaas.clientUsers 0 | quote }} - name: SASL_USER_PASSWORD valueFrom: secretKeyRef: name: {{ include "kafka.jaasSecretName" . }} - key: client-password + key: client-passwords {{- end }} ports: - name: metrics diff --git a/bitnami/kafka/templates/statefulset.yaml b/bitnami/kafka/templates/statefulset.yaml index 33843a0d3..36dfdbee9 100644 --- a/bitnami/kafka/templates/statefulset.yaml +++ b/bitnami/kafka/templates/statefulset.yaml @@ -161,6 +161,12 @@ spec: {{- else }} value: "INTERNAL:{{ $interBrokerProtocol }},CLIENT:{{ $clientProtocol }}" {{- end }} + {{- if or ($clientProtocol | regexFind "SASL") ($interBrokerProtocol | regexFind "SASL") .Values.auth.jaas.zookeeperUser }} + - name: KAFKA_CFG_SASL_ENABLED_MECHANISMS + value: {{ include "kafka.auth.saslMechanisms" ( dict "type" .Values.auth.saslMechanisms ) }} + - name: KAFKA_CFG_SASL_MECHANISM_INTER_BROKER_PROTOCOL + value: {{ upper .Values.auth.saslInterBrokerMechanism | quote }} + {{- end }} - name: KAFKA_CFG_LISTENERS {{- if .Values.listeners }} value: {{ .Values.listeners }} @@ -188,13 +194,13 @@ spec: - name: KAFKA_OPTS value: "-Djava.security.auth.login.config=/opt/bitnami/kafka/conf/kafka_jaas.conf" {{- if (include "kafka.client.saslAuthentication" .) }} - - name: KAFKA_CLIENT_USER - value: {{ .Values.auth.jaas.clientUser | quote }} - - name: KAFKA_CLIENT_PASSWORD + - name: KAFKA_CLIENT_USERS + value: {{ join "," .Values.auth.jaas.clientUsers | quote }} + - name: KAFKA_CLIENT_PASSWORDS valueFrom: secretKeyRef: name: {{ include "kafka.jaasSecretName" . }} - key: client-password + key: client-passwords {{- end }} {{- if .Values.auth.jaas.zookeeperUser }} - name: KAFKA_ZOOKEEPER_PROTOCOL diff --git a/bitnami/kafka/values-production.yaml b/bitnami/kafka/values-production.yaml index d1a2acfe7..82ddc0544 100644 --- a/bitnami/kafka/values-production.yaml +++ b/bitnami/kafka/values-production.yaml @@ -244,6 +244,13 @@ auth: clientProtocol: sasl interBrokerProtocol: sasl + ## Allowed SASL mechanisms when clientProtocol or interBrokerProtocol are using either sasl or sasl_tls + ## + saslMechanisms: plain,scram-sha-256,scram-sha-512 + ## SASL mechanism for inter broker communication + ## + saslInterBrokerMechanism: plain + ## Name of the existing secret containing the truststore and ## one keystore per Kafka broker you have in the Kafka cluster. ## MANDATORY when 'tls', 'mtls', or 'sasl_tls' authentication protocols are used. @@ -271,13 +278,22 @@ auth: ## MANDATORY when method is 'sasl', or 'sasl_tls' ## jaas: - ## Kafka client user + ## Kafka client user list ## - clientUser: user + ## clientUsers: + ## - user1 + ## - user2 + ## + clientUsers: + - user - ## Kafka client password + ## Kafka client passwords ## - clientPassword: "" + ## clientPasswords: + ## - password1 + ## - password2 + ## + clientPasswords: [] ## Kafka inter broker communication user ## @@ -295,9 +311,9 @@ auth: ## zookeeperPassword: zookeeperPassword - ## Name of the existing secret containing credentials for clientUser, interBrokerUser and zookeeperUser. + ## Name of the existing secret containing credentials for clientUsers, interBrokerUser and zookeeperUser. ## Create this secret running the command below where SECRET_NAME is the name of the secret you want to create: - ## kubectl create secret generic SECRET_NAME --from-literal=client-password=CLIENT_PASSWORD --from-literal=inter-broker-password=INTER_BROKER_PASSWORD --from-literal=zookeeper-password=ZOOKEEPER_PASSWORD + ## kubectl create secret generic SECRET_NAME --from-literal=client-password=CLIENT_PASSWORD1,CLIENT_PASSWORD2 --from-literal=inter-broker-password=INTER_BROKER_PASSWORD --from-literal=zookeeper-password=ZOOKEEPER_PASSWORD ## # existingSecret: diff --git a/bitnami/kafka/values.yaml b/bitnami/kafka/values.yaml index 5049aecb5..7b9d44676 100644 --- a/bitnami/kafka/values.yaml +++ b/bitnami/kafka/values.yaml @@ -244,6 +244,13 @@ auth: clientProtocol: plaintext interBrokerProtocol: plaintext + ## Allowed SASL mechanisms when clientProtocol or interBrokerProtocol are using either sasl or sasl_tls + ## + saslMechanisms: plain,scram-sha-256,scram-sha-512 + ## SASL mechanism for inter broker communication + ## + saslInterBrokerMechanism: plain + ## Name of the existing secret containing the truststore and ## one keystore per Kafka broker you have in the Kafka cluster. ## MANDATORY when 'tls', 'mtls', or 'sasl_tls' authentication protocols are used. @@ -271,13 +278,22 @@ auth: ## MANDATORY when method is 'sasl', or 'sasl_tls' ## jaas: - ## Kafka client user + ## Kafka client user list ## - clientUser: user + ## clientUsers: + ## - user1 + ## - user2 + ## + clientUsers: + - user - ## Kafka client password + ## Kafka client passwords. This is mandatory if more than one user is specified in clientUsers. ## - clientPassword: "" + ## clientPasswords: + ## - password1 + ## - password2" + ## + clientPasswords: [] ## Kafka inter broker communication user ## @@ -295,9 +311,9 @@ auth: ## # zookeeperPassword: - ## Name of the existing secret containing credentials for clientUser, interBrokerUser and zookeeperUser. + ## Name of the existing secret containing credentials for clientUsers, interBrokerUser and zookeeperUser. ## Create this secret running the command below where SECRET_NAME is the name of the secret you want to create: - ## kubectl create secret generic SECRET_NAME --from-literal=client-password=CLIENT_PASSWORD --from-literal=inter-broker-password=INTER_BROKER_PASSWORD --from-literal=zookeeper-password=ZOOKEEPER_PASSWORD + ## kubectl create secret generic SECRET_NAME --from-literal=client-passwords=CLIENT_PASSWORD1,CLIENT_PASSWORD2 --from-literal=inter-broker-password=INTER_BROKER_PASSWORD --from-literal=zookeeper-password=ZOOKEEPER_PASSWORD ## # existingSecret: