Commit 2d44c63b authored by Waqar Ahmed's avatar Waqar Ahmed

Add a testing chart for ix-chart

parent 76b77b75
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/
dependencies:
- name: common
repository: file://../../../library/common/2104.0.0
version: 2104.0.0
digest: sha256:f0aa221073aafcc5e1602c2a9acb1a508ce72f6847c33dd4a9f9fe10017d5009
generated: "2021-04-08T16:09:30.065733+05:00"
apiVersion: v2
name: ix-chart-test
description: A Helm chart for deploying simple workloads Kubernetes
# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 2105.0.0
# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
appVersion: v1
dependencies:
- name: common
repository: file://../../../library/common/2104.0.0
version: 2104.0.0
# iX Chart
iX-chart is a chart designed to let user deploy a docker image in a TrueNAS SCALE kubernetes cluster.
It provides a mechanism to specify workload type, add external host interfaces in the pods, configure volumes and allocate host resources to the workload.
This chart will deploy a docker image as a kubernetes workload allowing user to configure the workload deployment / management.
## Introduction
iX-chart is designed for simple single docker image deployments.
## Configuration
The following table lists the configurable parameters of the iX chart and
their default values.
| Parameter | Description | Default |
|:-------------------------------|:----------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------|
| `workloadType` | Specify type of workload to deploy | `Deployment` |
| `cronSchedule` | Specify schedule for cronjob if `workloadType` is `CronJob`. | `{"minute": "5", "hour": "*", "dom": "*", "month": "*", "dow": "*"}` |
| `image.repository` | The image repository to pull from | `debian` |
| `image.tag` | The image tag to pull from | `latest` |
| `image.pullPolicy` | Image pull policy | `IfNotPresent` |
| `updateStrategy` | Upgrade Policy | `RollingUpdate` |
| `jobRestartPolicy` | Restart Policy for job type workload ( only applicable if `workloadType` is `Job`/`CronJob` | `OnFailure` |
| `containerCommand` | Commands to execute inside container overriding image CMD default | `null` |
| `containerArgs` | Specify arguments for container command | `null` |
| `containerEnvironmentVariables`| Container Environment Variables | `null` |
| `externalInterfaces` | Add external interfaces in the pod | `null` |
| `dnsPolicy` | Specify DNS Policy for pod | `Default` |
| `dnsConfig` | Specify custom DNS configuration which will be applied to the pod | `{"nameservers": [], "searches": []}` |
| `hostNetwork` | Use host network for workload ( similar to docker host networking ) | `false` |
| `hostPortsList` | Specify ports of host and workload to forward traffic from host port to workload port. | `null` |
| `portForwardingList` | Specify ports of node and workload to forward traffic from node port to workload port | `null` |
| `hostPathVolumes` | Specify host paths to be used as hostpath volumes for the workload | `null` |
| `volumes` | Specify `ix_volumes` | `null` |
| `livenessProbe` | Configure Liveness Probe for workload | `null` |
| `gpuConfiguration` | Allocate GPU to workload ( if available ) | `{}` |
## Persistence
Chart release iX chart offers 2 ways to have persistent storage:
1) `hostPathVolumes`
2) `volumes`
For (1), they are kubernetes host path volumes which the user can assign to the workload with RO/RW permissions.
(2) is a host path volume as well but it operates differently then (1) in terms of where it lives and how it's lifecycle is tied to the chart release.
For (2), users specify where they would like persistent storage in the workload and a dataset name ( it should be unique per each chart release ), based on this input,
system will create a dataset and then use it as a host path volume for the workload. During upgrades, snapshot will be taken for these volumes and on rollback users can subsequently
restore the snapshots hence the data.
When a chart release will be deleted, all (2) volumes data will be deleted unlike (1) ( until of course they are not in the chart release's dataset path ).
## Recommended Persistence Configuration Examples
The following is a recommended configuration example for creating ix volumes
```
"volumes": [
{
"datasetName": "ix_volume1",
"mountPath": "/mount_test1"
},
{
"datasetName": "ix_volume2",
"mountPath": "/mount_test2"
}
]
```
`mountPath` refers to the path inside the pod.
---
The following is a recommended configuration example for `hostPathVolumes`
```
"hostPathVolumes": [
{
"hostPath": "/mnt/pool/test_dir",
"mountPath": "/test_dir",
"readOnly": true
},
{
"hostPath": "/mnt/pool/test_file",
"mountPath": "/test_file",
"readOnly": true
}
]
```
## External Interfaces Configuration Examples
```
"externalInterfaces": [
{
"hostInterface": "ens3",
"ipam": {"type": "dhcp"}
},
{
"hostInterface": "br0",
"ipam": {
"type": "static",
"staticIPConfigurations": ["192.168.0.120/24"],
"staticRoutes": [
{"destination": "0.0.0.0/0", "gateway": "192.168.0.1"}
]
}
}
]
```
The first entry in the above configuration example will add an external interface in the pod using macvlan with `ipam` being set to `dhcp`.
Second entry will add an external interface in the pod using `bridge` and using static IP configuration for the external interface.
# iX-Chart
iX-chart is a chart designed to let user deploy a docker image in a TrueNAS SCALE kubernetes cluster.
It provides a mechanism to specify workload type, add external host interfaces in the pods, configure volumes and allocate host resources to the workload.
# Default values for ix-chart.
image:
repository: debian
pullPolicy: IfNotPresent
# Overrides the image tag whose default is the chart appVersion.
tag: "latest"
# Restart / Update policy
updateStrategy: RollingUpdate
# Container CMD / entrypoint
containerCommand: []
containerArgs: []
containerEnvironmentVariables: []
# Network related configuration
externalInterfaces: []
portForwardingList: []
hostNetwork: false
dnsPolicy: Default
dnsConfig:
nameservers: []
searches: []
# Storage related configuration
hostPathVolumes: []
volumes: []
# Probes
# Liveness Probe
livenessProbe: null
# Workload type
workloadType: "Deployment"
gpuConfiguration: {}
This diff is collapsed.
1. Get the application URL by running these commands:
{{/*
Container Command
*/}}
{{- define "containerCommand" }}
{{- if .Values.containerCommand }}
command:
{{- range .Values.containerCommand }}
- {{ . | quote}}
{{- end }}
{{- end }}
{{- end }}
{{/*
Container Args
*/}}
{{- define "containerArgs" }}
{{- if .Values.containerArgs }}
args:
{{- range .Values.containerArgs }}
- {{ . | quote}}
{{- end }}
{{- end }}
{{- end }}
{{/*
Container Environment Variables
*/}}
{{- define "containerEnvVariables" }}
{{- if .Values.containerEnvironmentVariables }}
env:
{{- range .Values.containerEnvironmentVariables }}
- name: {{ .name | quote }}
value: {{ .value | quote }}
{{- end }}
{{- end }}
{{- end }}
{{/*
Container Liveness Probe
*/}}
{{- define "containerLivenssProbe" }}
{{- if .Values.livenessProbe }}
livenessProbe:
exec:
command:
{{ toYaml .Values.livenessProbe.command | indent 16 }}
initialDelaySeconds: {{ .Values.livenessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.periodSeconds }}
{{- end }}
{{- end }}
{{/*
Container Ports
*/}}
{{- define "containerPorts" }}
{{- if or .Values.portForwardingList .Values.hostPortsList }}
ports:
{{- range $index, $config := .Values.portForwardingList }}
- containerPort: {{ $config.containerPort }}
{{- end }}
{{- range $index, $config := .Values.hostPortsList }}
- containerPort: {{ $config.containerPort }}
hostPort: {{ $config.hostPort }}
{{- end }}
{{- end }}
{{- end }}
{{/*
Container Resource Configuration
*/}}
{{- define "containerResourceConfiguration" }}
{{- if .Values.gpuConfiguration }}
resources:
limits:
{{- toYaml .Values.gpuConfiguration | nindent 4 }}
{{- end }}
{{- end }}
{{/* vim: set filetype=mustache: */}}
{{/*
Expand the name of the chart.
*/}}
{{- define "ix-chart.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "ix-chart.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "ix-chart.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Common labels
*/}}
{{- define "ix-chart.labels" -}}
helm.sh/chart: {{ include "ix-chart.chart" . }}
{{ include "ix-chart.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}
{{/*
Selector labels
*/}}
{{- define "ix-chart.selectorLabels" -}}
app.kubernetes.io/name: {{ include "ix-chart.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
{{/*
Create the name of the service account to use
*/}}
{{- define "ix-chart.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "ix-chart.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}
{{/*
Convert cron schema object to cron format
*/}}
{{- define "cronExpression" }}
{{- printf "%s %s %s %s %s " .minute .hour .dom .month .dow }}
{{- end }}
{{/*
DNS Configuration
*/}}
{{- define "dnsConfiguration" }}
dnsPolicy: {{ .Values.dnsPolicy }}
{{- if .Values.dnsConfig }}
dnsConfig:
{{- toYaml .Values.dnsConfig | nindent 2 }}
{{- end }}
{{- end }}
{{/*
Volumes Configuration
*/}}
{{- define "volumeConfiguration" }}
{{- if or .Values.ixVolumes .Values.hostPathVolumes }}
volumes:
{{- range $index, $hostPathConfiguration := .Values.hostPathVolumes }}
- name: ix-host-path-{{ $.Release.Name }}-{{ $index }}
hostPath:
path: {{ $hostPathConfiguration.hostPath }}
{{- end }}
{{- range $index, $hostPathConfiguration := .Values.ixVolumes }}
{{ $dsName := base $hostPathConfiguration.hostPath }}
- name: ix-host-volume-{{ $.Release.Name }}-{{ $dsName }}
hostPath:
path: {{ $hostPathConfiguration.hostPath }}
{{- end }}
{{- end }}
{{- end }}
{{/*
Volume Mounts Configuration
*/}}
{{- define "volumeMountsConfiguration" }}
{{- if or .Values.hostPathVolumes .Values.ixVolumes }}
volumeMounts:
{{- range $index, $hostPathConfiguration := .Values.hostPathVolumes }}
- mountPath: {{ $hostPathConfiguration.mountPath }}
name: ix-host-path-{{ $.Release.Name }}-{{ $index }}
readOnly: {{ $hostPathConfiguration.readOnly }}
{{- end }}
{{- range $index, $hostPathConfiguration := .Values.volumes }}
- mountPath: {{ $hostPathConfiguration.mountPath }}
name: ix-host-volume-{{ $.Release.Name }}-{{ $hostPathConfiguration.datasetName }}
{{- end }}
{{- end }}
{{- end }}
{{/*
Check if workload type is a deployment
*/}}
{{- define "workloadIsDeployment" }}
{{- if eq .Values.workloadType "Deployment" }}
{{- true -}}
{{- else }}
{{- false -}}
{{- end }}
{{- end }}
{{/*
Check if workload type is a cronjob
*/}}
{{- define "workloadIsCronJob" }}
{{- if eq .Values.workloadType "CronJob" }}
{{- true -}}
{{- else }}
{{- false -}}
{{- end }}
{{- end }}
{{/*
Get API Version based on workload type
*/}}
{{- define "apiVersion" -}}
{{- if eq (include "workloadIsDeployment" .) "true" }}
{{- printf "apps/v1" }}
{{- else if eq (include "workloadIsCronJob" .) "true" }}
{{- printf "batch/v1beta1" }}
{{- else }}
{{- printf "batch/v1" }}
{{- end }}
{{- end }}
{{/*
Get Restart policy based on workload type
*/}}
{{- define "restartPolicy" -}}
{{- if eq (include "workloadIsDeployment" .) "true" }}
{{- print "Always" }}
{{- else }}
{{- printf "%s" .Values.jobRestartPolicy }}
{{- end }}
{{- end }}
{{/*
Pod specification
*/}}
{{- define "podSepc" }}
restartPolicy: {{ template "restartPolicy" . }}
hostNetwork: {{ .Values.hostNetwork }}
containers:
- name: {{ .Chart.Name }}
{{- include "volumeMountsConfiguration" . | indent 2}}
securityContext:
{{- toYaml .Values.securityContext | nindent 12 }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default "latest" }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
{{- include "containerCommand" . | indent 2 }}
{{- include "containerArgs" . | indent 2 }}
{{- include "containerEnvVariables" . | indent 2 }}
{{- include "containerLivenssProbe" . | indent 2 }}
{{- include "containerPorts" . | indent 2 }}
{{- include "containerResourceConfiguration" . | indent 2 }}
{{- include "volumeConfiguration" . }}
{{- include "dnsConfiguration" . }}
{{- end }}
{{/*
Annotations for workload
*/}}
{{- define "workloadAnnotations" }}
rollme: {{ randAlphaNum 5 | quote }}
{{- if .Values.ixExternalInterfacesConfigurationNames }}
k8s.v1.cni.cncf.io/networks: {{ join ", " .Values.ixExternalInterfacesConfigurationNames }}
{{- end }}
{{- end }}
{{/*
Metadata for workload
*/}}
{{- define "commonMetadataWorkload" }}
labels:
{{- include "ix-chart.selectorLabels" . | nindent 2 }}
annotations:
{{- include "workloadAnnotations" . | nindent 2 }}
{{- end }}
{{/*
Deployment Spec
*/}}
{{- define "deploymentSpec" }}
strategy:
{{- if and (eq .Values.updateStrategy "RollingUpdate") .Values.hostPortsList }}
{{- fail "RollingUpdate is not allowed when host ports are specified" }}
{{- end }}
type: {{ .Values.updateStrategy }}
selector:
matchLabels:
{{- include "ix-chart.selectorLabels" . | nindent 4 }}
template:
metadata:
{{ include "commonMetadataWorkload" . | nindent 4 }}
spec:
{{- include "podSepc" . | indent 4 }}
{{- end }}
{{/*
Job Spec Common
*/}}
{{- define "jobSpecCommon" }}
metadata:
{{ include "commonMetadataWorkload" . | nindent 4 }}
spec:
{{- include "podSepc" . | indent 2 }}
{{- end }}
{{/*
Job Spec
*/}}
{{- define "jobSpec" }}
template:
{{ include "jobSpecCommon" . | nindent 2 }}
{{- end }}
{{/*
CronJob Spec
*/}}
{{- define "cronJobSpec" }}
schedule: {{ include "cronExpression" .Values.cronSchedule | quote }}
jobTemplate:
spec:
{{ include "jobSpec" . | nindent 4 }}
{{- end }}
{{- range $index, $iface := .Values.ixExternalInterfacesConfiguration }}
---
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
name: ix-{{ $.Release.Name }}-{{ $index }}
spec:
config: '{{ $iface }}'
{{- end }}
{{- if .Values.portForwardingList }}
apiVersion: v1
kind: Service
metadata:
name: {{ include "ix-chart.fullname" . }}
labels:
{{- include "ix-chart.labels" . | nindent 4 }}
spec:
type: NodePort
ports:
{{- range $index, $config := .Values.portForwardingList }}
- port: {{ $config.containerPort }}
targetPort: {{ $config.containerPort }}
protocol: {{ $config.protocol }}
nodePort: {{ $config.nodePort }}
name: ix-{{ $.Release.Name }}-{{ $config.nodePort }}-{{ $index }}
{{- end }}
selector:
{{- include "ix-chart.selectorLabels" . | nindent 4 }}
{{- end }}
\ No newline at end of file
apiVersion: v1
kind: Pod
metadata:
name: "{{ .Release.Name }}-deployment-test"
annotations:
"helm.sh/hook": test
spec:
containers:
- name: {{ .Release.Name }}-deployment-test
image: "busybox"
command:
- nc
args:
- "-vz"
- "{{ include "ix-chart.fullname" . }}"
- "80"
restartPolicy: Never
apiVersion: {{ template "apiVersion" . }}
kind: {{ .Values.workloadType }}
metadata:
name: {{ include "ix-chart.fullname" . }}
labels:
{{- include "ix-chart.labels" . | nindent 4 }}
spec:
{{- if eq (include "workloadIsDeployment" .) "true" }}
{{ include "deploymentSpec" . | nindent 2 }}
{{- else if eq (include "workloadIsCronJob" .) "true" }}
{{ include "cronJobSpec" . | nindent 2 }}
{{- else }}
{{ include "jobSpec" . | nindent 2 }}
{{- end }}
# Default values for ix-chart.
image:
repository: nginx
pullPolicy: IfNotPresent
# Overrides the image tag whose default is the chart appVersion.
tag: latest
# Restart / Update policy
updateStrategy: RollingUpdate
# Container CMD / entrypoint
containerCommand: []
containerArgs: []
containerEnvironmentVariables: []
# Network related configuration
externalInterfaces: []
portForwardingList: [{"containerPort": 80, "nodePort": 32000}]
hostPortsList: []
hostNetwork: false
dnsPolicy: Default
dnsConfig:
nameservers: []
searches: []
# Storage related configuration
hostPathVolumes: []
volumes: []
# Probes
# Liveness Probe
livenessProbe: null
# Workload type
workloadType: "Deployment"
gpuConfiguration: {}
categories:
- generic
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment