[MinIO](https://min.io) is a High Performance Object Storage released under Apache License v2.0. It is API compatible with Amazon S3 cloud storage service. Use MinIO to build high performance infrastructure for machine learning, analytics and application data workloads.
MinIO supports [distributed mode](https://docs.minio.io/docs/distributed-minio-quickstart-guide). In distributed mode, you can pool multiple drives (even on different machines) into a single object storage server.
For more detailed documentation please visit [here](https://docs.minio.io/)
Introduction
------------
This chart bootstraps MinIO deployment on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager.
Prerequisites
-------------
- Kubernetes 1.4+ with Beta APIs enabled for default standalone mode.
- Kubernetes 1.5+ with Beta APIs enabled to run MinIO in [distributed mode](#distributed-minio).
- PV provisioner support in the underlying infrastructure.
The command deploys MinIO on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation.
### Release name
An instance of a chart running in a Kubernetes cluster is called a release. Each release is identified by a unique name within the cluster. Helm automatically assigns a unique release name after installing the chart. You can also set your preferred name by:
```bash
$ helm install my-release minio/minio
```
### Access and Secret keys
By default a pre-generated access and secret key will be used. To override the default keys, pass the access and secret keys as arguments to helm install.
[ConfigMap](https://kubernetes.io/docs/user-guide/configmap/) allows injecting containers with configuration data even while a Helm release is deployed.
To update your MinIO server configuration while it is deployed in a release, you need to
1. Check all the configurable values in the MinIO chart using `helm inspect values minio/minio`.
2. Override the `minio_server_config` settings in a YAML formatted file, and then pass that file like this `helm upgrade -f config.yaml minio/minio`.
3. Restart the MinIO server(s) for the changes to take effect.
You can also check the history of upgrades to a release using `helm history my-release`. Replace `my-release` with the actual release name.
### Installing certificates from third party CAs
MinIO can connect to other servers, including MinIO nodes or other server types such as NATs and Redis. If these servers use certificates that were not registered with a known CA, add trust for these certificates to MinIO Server by bundling these certificates into a Kubernetes secret and providing it to Helm via the `trustedCertsSecret` value. If `.Values.tls.enabled` is `true` and you're installing certificates for third party CAs, remember to include Minio's own certificate with key `public.crt`, if it also needs to be trusted.
For instance, given that TLS is enabled and you need to add trust for Minio's own CA and for the CA of a Keycloak server, a Kubernetes secret can be created from the certificate files using `kubectl`:
| `priorityClassName` | Pod priority settings | `""` |
| `securityContext.enabled` | Enable to run containers as non-root. NOTE: if `persistence.enabled=false` then securityContext will be automatically disabled | `true` |
| `securityContext.runAsUser` | User id of the user for the container | `1000` |
| `securityContext.runAsGroup` | Group id of the user for the container | `1000` |
| `securityContext.fsGroup` | Group id of the persistent volume mount for the container | `1000` |
| `nodeSelector` | Node labels for pod assignment | `{}` |
| `affinity` | Affinity settings for pod assignment | `{}` |
| `tolerations` | Toleration labels for pod assignment | `[]` |
| `podAnnotations` | Pod annotations | `{}` |
| `podLabels` | Pod Labels | `{}` |
| `tls.enabled` | Enable TLS for MinIO server | `false` |
| `tls.certSecret` | Kubernetes Secret with `public.crt` and `private.key` files. | `""` |
| `defaultBucket.enabled` | If set to true, a bucket will be created after MinIO install | `false` |
| `updatePrometheusJob.podAnnotations` | Additional annotations for the pods of the Kubernetes Batch (update-prometheus-secret) | `""` |
| `updatePrometheusJob.securityContext.enabled` | Enable to run Kubernetes Batch (update-prometheus-secret) containers as non-root. | `false` |
| `updatePrometheusJob.securityContext.runAsUser` | User id of the user for the container | `1000` |
| `updatePrometheusJob.securityContext.runAsGroup` | Group id of the user for the container | `1000` |
| `updatePrometheusJob.securityContext.fsGroup` | Group id of the persistent volume mount for the container | `1000` |
| `s3gateway.enabled` | Use MinIO as a [s3 gateway](https://github.com/minio/minio/blob/master/docs/gateway/s3.md) | `false` |
| `s3gateway.replicas` | Number of s3 gateway instances to run in parallel | `4` |
| `s3gateway.serviceEndpoint` | Endpoint to the S3 compatible service | `""` |
| `s3gateway.accessKey` | Access key of S3 compatible service | `""` |
| `s3gateway.secretKey` | Secret key of S3 compatible service | `""` |
| `azuregateway.enabled` | Use MinIO as an [azure gateway](https://docs.minio.io/docs/minio-gateway-for-azure) | `false` |
| `azuregateway.replicas` | Number of azure gateway instances to run in parallel | `4` |
| `gcsgateway.enabled` | Use MinIO as a [Google Cloud Storage gateway](https://docs.minio.io/docs/minio-gateway-for-gcs) | `false` |
| `gcsgateway.gcsKeyJson` | credential json file of service account key | `""` |
| `gcsgateway.projectId` | Google cloud project id | `""` |
| `nasgateway.enabled` | Use MinIO as a [NAS gateway](https://docs.MinIO.io/docs/minio-gateway-for-nas) | `false` |
| `nasgateway.replicas` | Number of NAS gateway instances to be run in parallel on a PV | `4` |
| `environment` | Set MinIO server relevant environment variables in `values.yaml` file. MinIO containers will be passed these variables when they start. | `MINIO_STORAGE_CLASS_STANDARD: EC:4"` |
| `metrics.serviceMonitor.enabled` | Set this to `true` to create ServiceMonitor for Prometheus operator | `false` |
| `metrics.serviceMonitor.additionalLabels` | Additional labels that can be used so ServiceMonitor will be discovered by Prometheus | `{}` |
| `metrics.serviceMonitor.namespace` | Optional namespace in which to create ServiceMonitor | `nil` |
| `metrics.serviceMonitor.interval` | Scrape interval. If not set, the Prometheus default scrape interval is used | `nil` |
| `metrics.serviceMonitor.scrapeTimeout` | Scrape timeout. If not set, the Prometheus default scrape timeout is used | `nil` |
| `etcd.endpoints` | Enpoints of etcd | `[]` |
| `etcd.pathPrefix` | Prefix for all etcd keys | `""` |
| `etcd.corednsPathPrefix` | Prefix for all CoreDNS etcd keys | `""` |
| `etcd.clientCert` | Certificate used for SSL/TLS connections to etcd [(etcd Security)](https://etcd.io/docs/latest/op-guide/security/) | `""` |
| `etcd.clientCertKey` | Key for the certificate [(etcd Security)](https://etcd.io/docs/latest/op-guide/security/) | `""` |
Some of the parameters above map to the env variables defined in the [MinIO DockerHub image](https://hub.docker.com/r/minio/minio/).
You can specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example,
> **Tip**: You can use the default [values.yaml](minio/values.yaml)
Distributed MinIO
-----------
This chart provisions a MinIO server in standalone mode, by default. To provision MinIO server in [distributed mode](https://docs.minio.io/docs/distributed-minio-quickstart-guide), set the `mode` field to `distributed`,
```bash
$ helm install--setmode=distributed minio/minio
```
This provisions MinIO server in distributed mode with 4 nodes. To change the number of nodes in your distributed MinIO server, set the `replicas` field,
This provisions MinIO server in distributed mode with 8 nodes. Note that the `replicas` value should be a minimum value of 4, there is no limit on number of servers you can run.
You can also expand an existing deployment by adding new zones, following command will create a total of 16 nodes with each zone running 8 nodes.
### StatefulSet [limitations](http://kubernetes.io/docs/concepts/abstractions/controllers/statefulsets/#limitations) applicable to distributed MinIO
1. StatefulSets need persistent storage, so the `persistence.enabled` flag is ignored when `mode` is set to `distributed`.
2. When uninstalling a distributed MinIO release, you'll need to manually delete volumes associated with the StatefulSet.
NAS Gateway
-----------
### Prerequisites
MinIO in [NAS gateway mode](https://docs.minio.io/docs/minio-gateway-for-nas) can be used to create multiple MinIO instances backed by single PV in `ReadWriteMany` mode. Currently few [Kubernetes volume plugins](https://kubernetes.io/docs/user-guide/persistent-volumes/#access-modes) support `ReadWriteMany` mode. To deploy MinIO NAS gateway with Helm chart you'll need to have a Persistent Volume running with one of the supported volume plugins. [This document](https://kubernetes.io/docs/user-guide/volumes/#nfs)
outlines steps to create a NFS PV in Kubernetes cluster.
### Provision NAS Gateway MinIO instances
To provision MinIO servers in [NAS gateway mode](https://docs.minio.io/docs/minio-gateway-for-nas), set the `nasgateway.enabled` field to `true`,
This provisions 4 MinIO NAS gateway instances backed by single storage. To change the number of instances in your MinIO deployment, set the `replicas` field,
This provisions MinIO NAS gateway with 8 instances.
Persistence
-----------
This chart provisions a PersistentVolumeClaim and mounts corresponding persistent volume to default location `/export`. You'll need physical storage available in the Kubernetes cluster for this to work. If you'd rather use `emptyDir`, disable PersistentVolumeClaim by:
> *"An emptyDir volume is first created when a Pod is assigned to a Node, and exists as long as that Pod is running on that node. When a Pod is removed from a node for any reason, the data in the emptyDir is deleted forever."*
Existing PersistentVolumeClaim
------------------------------
If a Persistent Volume Claim already exists, specify it during installation.
| `gcs_key.json` | `gcsgateway.gcsKeyJson` | GCS key if you are using the GCS gateway feature. Optional |
| `awsAccessKeyId` | `s3gateway.accessKey` | S3 access key if you are using the S3 gateway feature. Optional |
| `awsSecretAccessKey` | `s3gateway.secretKey` | S3 secret key if you are using the S3 gateway feature. Optional |
| `etcd_client_cert.pem` | `etcd.clientCert` | Certificate for SSL/TLS connections to etcd. Optional |
| `etcd_client_cert_key.pem` | `etcd.clientCertKey` | Corresponding key for certificate above. Mandatory when etcd certificate defined. |
All corresponding variables will be ignored in values file.
Configure TLS
-------------
To enable TLS for MinIO containers, acquire TLS certificates from a CA or create self-signed certificates. While creating / acquiring certificates ensure the corresponding domain names are set as per the standard [DNS naming conventions](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#pod-identity) in a Kubernetes StatefulSet (for a distributed MinIO setup). Then create a secret using
