[bitnami/minio] Get Azure Storage Account credentials via external secrets + Gateway autoscaling (#7317)

* Adding optional Azure Storage Account credential source

Enables Azure Storage Account credential extraction from external secrets specified in values.yaml

* Add Autoscaling to Minio Gateway

Add an HPA component
Add autoscaling configuration in values.yaml
Modify update strategy for Minio Gateway deployment to be more flexible

* Fix azure credentials values validation

Fix so that credentials fail if either there's no explicit credentials or if at least one of the existing secret references is empty

* Bump minor chart version for Minio

* Adjusting Minio gateway.updateStrategy type description

* Updating chart version

* Updating gateway autoscaling metadata

* Adjusting template formatting for compatibility

Co-authored-by: Ayan Bikalapov <abikalapov@explorance.com>
This commit is contained in:
aiceball
2021-08-31 10:39:55 +02:00
committed by GitHub
co-authored by Ayan Bikalapov
parent bc571361bf
commit 4040ce1d32
6 changed files with 104 additions and 22 deletions
+1 -1
View File
@@ -25,4 +25,4 @@ name: minio
sources:
- https://github.com/bitnami/bitnami-docker-minio
- https://min.io
version: 7.1.10
version: 7.2.0
+28 -19
View File
@@ -252,25 +252,34 @@ The command removes all the Kubernetes components associated with the chart and
### Gateway parameters
| Name | Description | Value |
| --------------------------------------- | -------------------------------------------------------------- | -------------------------- |
| `gateway.enabled` | Use MinIO&reg; as Gateway for other storage systems | `false` |
| `gateway.type` | Gateway type. Supported types are: `azure`, `gcs`, `nas`, `s3` | `s3` |
| `gateway.replicaCount` | Number of MinIO&reg; Gateway replicas | `4` |
| `gateway.auth.azure.accessKey` | Access Key to access MinIO using Azure Gateway | `""` |
| `gateway.auth.azure.secretKey` | Secret Key to access MinIO using Azure Gateway | `""` |
| `gateway.auth.azure.storageAccountName` | Azure Storage Account Name to use to access Azure Blob Storage | `""` |
| `gateway.auth.azure.storageAccountKey` | Azure Storage Account Key to use to access Azure Blob Storage | `""` |
| `gateway.auth.gcs.accessKey` | Access Key to access MinIO using GCS Gateway | `""` |
| `gateway.auth.gcs.secretKey` | Secret Key to access MinIO using GCS Gateway | `""` |
| `gateway.auth.gcs.keyJSON` | Service Account key to access GCS | `""` |
| `gateway.auth.gcs.projectID` | GCP Project ID to use | `""` |
| `gateway.auth.nas.accessKey` | Access Key to access MinIO using NAS Gateway | `""` |
| `gateway.auth.nas.secretKey` | Secret Key to access MinIO using NAS Gateway | `""` |
| `gateway.auth.s3.accessKey` | Access Key to use to access AWS S3 | `""` |
| `gateway.auth.s3.secretKey` | Secret Key to use to access AWS S3 | `""` |
| `gateway.auth.s3.serviceEndpoint` | AWS S3 endpoint | `https://s3.amazonaws.com` |
| Name | Description | Value |
|----------------------------------------------------------|----------------------------------------------------------------------------------------------|----------------------------|
| `gateway.enabled` | Use MinIO&reg; as Gateway for other storage systems | `false` |
| `gateway.type` | Gateway type. Supported types are: `azure`, `gcs`, `nas`, `s3` | `s3` |
| `gateway.replicaCount` | Number of MinIO&reg; Gateway replicas | `4` |
| `gateway.updateStrategy.type` | Update strategy type for MinIO&reg; Gateway replicas | `Recreate` |
| `gateway.autoscaling.enabled` | Enable autoscaling for MinIO&reg; Gateway deployment | `false` |
| `gateway.autoscaling.minReplicas` | Minimum number of replicas to scale back | `4` |
| `gateway.autoscaling.maxReplicas` | Maximum number of replicas to scale out | `4` |
| `gateway.autoscaling.targetCPU` | Target CPU utilization percentage | `""` |
| `gateway.autoscaling.targetMemory` | Target Memory utilization percentage | `""` |
| `gateway.auth.azure.accessKey` | Access Key to access MinIO using Azure Gateway | `""` |
| `gateway.auth.azure.secretKey` | Secret Key to access MinIO using Azure Gateway | `""` |
| `gateway.auth.azure.storageAccountName` | Azure Storage Account Name to use to access Azure Blob Storage | `""` |
| `gateway.auth.azure.storageAccountKey` | Azure Storage Account Key to use to access Azure Blob Storage | `""` |
| `gateway.auth.azure.storageAccountNameExistingSecret` | Existing Secret name to extract Azure Storage Account Name from to access Azure Blob Storage | `""` |
| `gateway.auth.azure.storageAccountNameExistingSecretKey` | Existing Secret key to extract Azure Storage Account Name from to access Azure Blob Storage | `""` |
| `gateway.auth.azure.storageAccountKeyExistingSecret` | Existing Secret name to extract Azure Storage Account Key from to access Azure Blob Storage | `""` |
| `gateway.auth.azure.storageAccountKeyExistingSecretKey` | Existing Secret key to extract Azure Storage Account Key from to access Azure Blob Storage | `""` |
| `gateway.auth.gcs.accessKey` | Access Key to access MinIO using GCS Gateway | `""` |
| `gateway.auth.gcs.secretKey` | Secret Key to access MinIO using GCS Gateway | `""` |
| `gateway.auth.gcs.keyJSON` | Service Account key to access GCS | `""` |
| `gateway.auth.gcs.projectID` | GCP Project ID to use | `""` |
| `gateway.auth.nas.accessKey` | Access Key to access MinIO using NAS Gateway | `""` |
| `gateway.auth.nas.secretKey` | Secret Key to access MinIO using NAS Gateway | `""` |
| `gateway.auth.s3.accessKey` | Access Key to use to access AWS S3 | `""` |
| `gateway.auth.s3.secretKey` | Secret Key to use to access AWS S3 | `""` |
| `gateway.auth.s3.serviceEndpoint` | AWS S3 endpoint | `https://s3.amazonaws.com` |
Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example,
+4 -1
View File
@@ -238,10 +238,13 @@ minio: gateway.type
Validate values of MinIO&reg; - when using MinIO&reg; as an Azure Gateway, the StorageAccount Name/Key are required
*/}}
{{- define "minio.validateValues.gateway.azure.credentials" -}}
{{- if and .Values.gateway.enabled (eq .Values.gateway.type "azure") (or (empty .Values.gateway.auth.azure.storageAccountName) (empty .Values.gateway.auth.azure.storageAccountKey)) }}
{{- if and .Values.gateway.enabled (eq .Values.gateway.type "azure") (or (empty .Values.gateway.auth.azure.storageAccountName) (empty .Values.gateway.auth.azure.storageAccountKey)) (or (empty .Values.gateway.auth.azure.storageAccountNameExistingSecret) (empty .Values.gateway.auth.azure.storageAccountNameExistingSecretKey) (empty .Values.gateway.auth.azure.storageAccountKeyExistingSecret) (empty .Values.gateway.auth.azure.storageAccountKeyExistingSecretKey)) }}
minio: gateway.auth.azure
The StorageAccount name and key are required to use MinIO&reg; as a Azure Gateway.
Please set a valid StorageAccount information (--set gateway.auth.azure.storageAccountName="xxxx",gateway.auth.azure.storageAccountKey="yyyy")
Alternatively, specify secrets info to extract StorageAccount name and key:
--set gateway.auth.azure.storageAccountNameExistingSecret="xxxx", --set gateway.auth.azure.storageAccountNameExistingSecretKey="yyyy",
--set gateway.auth.azure.storageAccountKeyExistingSecret="aaaa", --set gateway.auth.azure.storageAccountKeyExistingSecretKey="bbbb"
{{- end -}}
{{- end -}}
@@ -12,8 +12,12 @@ metadata:
annotations: {{- include "common.tplvalues.render" ( dict "value" .Values.commonAnnotations "context" $ ) | nindent 4 }}
{{- end }}
spec:
{{- if not .Values.gateway.autoscaling.enabled }}
replicas: {{ .Values.gateway.replicaCount }}
{{- if .Values.deployment.updateStrategy }}
{{- end }}
{{- if .Values.gateway.updateStrategy }}
strategy: {{- toYaml .Values.gateway.updateStrategy | nindent 4 }}
{{- else }}
strategy: {{- toYaml .Values.deployment.updateStrategy | nindent 4 }}
{{- end }}
selector:
@@ -96,13 +100,23 @@ spec:
- name: AZURE_STORAGE_ACCOUNT
valueFrom:
secretKeyRef:
{{- if not ( or (empty .Values.gateway.auth.azure.storageAccountNameExistingSecret) (empty .Values.gateway.auth.azure.storageAccountNameExistingSecretKey))}}
name: {{ .Values.gateway.auth.azure.storageAccountNameExistingSecret }}
key: {{ .Values.gateway.auth.azure.storageAccountNameExistingSecretKey }}
{{- else }}
name: {{ include "minio.secretName" . }}
key: azure-storage-account-name
{{- end }}
- name: AZURE_STORAGE_KEY
valueFrom:
secretKeyRef:
{{- if not ( or (empty .Values.gateway.auth.azure.storageAccountNameExistingSecret) (empty .Values.gateway.auth.azure.storageAccountNameExistingSecretKey))}}
name: {{ .Values.gateway.auth.azure.storageAccountKeyExistingSecret }}
key: {{ .Values.gateway.auth.azure.storageAccountKeyExistingSecretKey }}
{{- else }}
name: {{ include "minio.secretName" . }}
key: azure-storage-account-key
{{- end }}
{{- else if and (eq .Values.gateway.type "gcs") .Values.gateway.auth.gcs.keyJSON }}
- name: GOOGLE_APPLICATION_CREDENTIALS
value: "/opt/bitnami/minio/secrets/key.json"
+33
View File
@@ -0,0 +1,33 @@
{{- if .Values.gateway.autoscaling.enabled }}
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
name: {{ template "common.names.fullname" . }}
labels: {{- include "common.labels.standard" . | nindent 4 }}
{{- if .Values.commonLabels }}
{{- include "common.tplvalues.render" ( dict "value" .Values.commonLabels "context" $ ) | nindent 4 }}
{{- end }}
{{- if .Values.commonAnnotations }}
annotations: {{- include "common.tplvalues.render" ( dict "value" .Values.commonAnnotations "context" $ ) | nindent 4 }}
{{- end }}
spec:
scaleTargetRef:
apiVersion: {{ include "common.capabilities.deployment.apiVersion" . }}
kind: Deployment
name: {{ template "common.names.fullname" . }}
minReplicas: {{ .Values.gateway.autoscaling.minReplicas }}
maxReplicas: {{ .Values.gateway.autoscaling.maxReplicas }}
metrics:
{{- if .Values.gateway.autoscaling.targetCPU }}
- type: Resource
resource:
name: cpu
targetAverageUtilization: {{ .Values.gateway.autoscaling.targetCPU }}
{{- end }}
{{- if .Values.gateway.autoscaling.targetMemory }}
- type: Resource
resource:
name: memory
targetAverageUtilization: {{ .Values.gateway.autoscaling.targetMemory }}
{{- end }}
{{- end }}
+23
View File
@@ -684,6 +684,21 @@ gateway:
## @param gateway.replicaCount Number of MinIO&reg; Gateway replicas
##
replicaCount: 4
## @param gateway.updateStrategy.type Update strategy type for MinIO&reg; Gateway replicas
updateStrategy:
type: Recreate
## Autoscaling configuration for MinIO&reg; Gateway. overrides gateway.replicaCount if enabled
## @param gateway.autoscaling.enabled Enable autoscaling for MinIO&reg; Gateway deployment
## @param gateway.autoscaling.minReplicas Minimum number of replicas to scale back
## @param gateway.autoscaling.maxReplicas Maximum number of replicas to scale out
## @param gateway.autoscaling.targetCPU Target CPU utilization percentage
## @param gateway.autoscaling.targetMemory Target Memory utilization percentage
autoscaling:
enabled: false
minReplicas: "4"
maxReplicas: "4"
targetCPU: ""
targetMemory: ""
## Gateway authentication configuration
##
auth:
@@ -692,12 +707,20 @@ gateway:
## @param gateway.auth.azure.secretKey Secret Key to access MinIO using Azure Gateway
## @param gateway.auth.azure.storageAccountName Azure Storage Account Name to use to access Azure Blob Storage
## @param gateway.auth.azure.storageAccountKey Azure Storage Account Key to use to access Azure Blob Storage
## @param gateway.auth.azure.storageAccountNameExistingSecret Existing Secret name to extract Azure Storage Account Name from to access Azure Blob Storage
## @param gateway.auth.azure.storageAccountNameExistingSecretKey Existing Secret key to extract Azure Storage Account Name from to use to access Azure Blob Storage
## @param gateway.auth.azure.storageAccountKeyExistingSecret Existing Secret name to extract Azure Storage Account Key from to access Azure Blob Storage
## @param gateway.auth.azure.storageAccountKeyExistingSecretKey Existing Secret key to extract Azure Storage Account Key from to use to access Azure Blob Storage
##
azure:
accessKey: ""
secretKey: ""
storageAccountName: ""
storageAccountKey: ""
storageAccountNameExistingSecret: ""
storageAccountNameExistingSecretKey: ""
storageAccountKeyExistingSecret: ""
storageAccountKeyExistingSecretKey: ""
## Authentication configuration for GCS. Ignored unless type=gcs
## @param gateway.auth.gcs.accessKey Access Key to access MinIO using GCS Gateway
## @param gateway.auth.gcs.secretKey Secret Key to access MinIO using GCS Gateway