Helm 与 Kustomize:Kubernetes 应用包管理实战
1. 为什么需要包管理工具
原始 YAML 的痛点:
1 2 3 4
| 问题 1:多环境配置重复(dev/staging/prod 几乎相同但有差异) 问题 2:版本管理困难(无法回滚到历史版本) 问题 3:依赖管理缺失(应用依赖其他组件) 问题 4:参数化困难(硬编码的镜像版本、副本数等)
|
2. Helm 深度解析
2.1 核心概念
1 2 3 4 5
| Chart:Helm 包(类似 apt/yum 的包) Release:Chart 的一次部署实例 Repository:Chart 仓库 Values:Chart 的配置参数 Template:Go 模板文件
|
2.2 Chart 目录结构
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
| my-app/ ├── Chart.yaml ├── values.yaml ├── values-prod.yaml ├── charts/ ├── templates/ │ ├── _helpers.tpl │ ├── deployment.yaml │ ├── service.yaml │ ├── ingress.yaml │ ├── configmap.yaml │ ├── hpa.yaml │ ├── serviceaccount.yaml │ └── NOTES.txt └── .helmignore
|
2.3 Chart.yaml
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
| apiVersion: v2 name: my-app description: My Application Helm Chart type: application version: 1.2.3 appVersion: "2.0.0"
dependencies: - name: postgresql version: "12.x.x" repository: https://charts.bitnami.com/bitnami condition: postgresql.enabled - name: redis version: "17.x.x" repository: https://charts.bitnami.com/bitnami condition: redis.enabled
|
2.4 values.yaml 设计
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62
| global: imageRegistry: "" imagePullSecrets: []
replicaCount: 2
image: repository: my-registry/my-app tag: "" pullPolicy: IfNotPresent
service: type: ClusterIP port: 80 targetPort: 8080
ingress: enabled: false className: nginx annotations: {} hosts: - host: my-app.example.com paths: - path: / pathType: Prefix tls: []
resources: requests: cpu: 100m memory: 128Mi limits: cpu: 500m memory: 512Mi
autoscaling: enabled: false minReplicas: 2 maxReplicas: 10 targetCPUUtilizationPercentage: 70
env: {}
config: LOG_LEVEL: info APP_ENV: production
postgresql: enabled: true auth: database: myapp username: myapp
|
2.5 模板编写
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43
| apiVersion: apps/v1 kind: Deployment metadata: name: {{ include "my-app.fullname" . }} labels: {{- include "my-app.labels" . | nindent 4 }} spec: {{- if not .Values.autoscaling.enabled }} replicas: {{ .Values.replicaCount }} {{- end }} selector: matchLabels: {{- include "my-app.selectorLabels" . | nindent 6 }} template: metadata: labels: {{- include "my-app.selectorLabels" . | nindent 8 }} annotations: checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }} spec: {{- with .Values.imagePullSecrets }} imagePullSecrets: {{- toYaml . | nindent 8 }} {{- end }} containers: - name: {{ .Chart.Name }} image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}" imagePullPolicy: {{ .Values.image.pullPolicy }} ports: - name: http containerPort: {{ .Values.service.targetPort }} env: {{- range $key, $value := .Values.env }} - name: {{ $key }} value: {{ $value | quote }} {{- end }} envFrom: - configMapRef: name: {{ include "my-app.fullname" . }} resources: {{- toYaml .Values.resources | nindent 10 }}
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
| {{/* 应用全名 */}} {{- define "my-app.fullname" -}} {{- if .Values.fullnameOverride }} {{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }} {{- else }} {{- $name := default .Chart.Name .Values.nameOverride }} {{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }} {{- end }} {{- end }}
{{/* 通用标签 */}} {{- define "my-app.labels" -}} helm.sh/chart: {{ include "my-app.chart" . }} {{ include "my-app.selectorLabels" . }} app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} app.kubernetes.io/managed-by: {{ .Release.Service }} {{- end }}
{{/* 选择器标签 */}} {{- define "my-app.selectorLabels" -}} app.kubernetes.io/name: {{ include "my-app.name" . }} app.kubernetes.io/instance: {{ .Release.Name }} {{- end }}
|
2.6 Helm 常用命令
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40
| helm repo add bitnami https://charts.bitnami.com/bitnami helm repo update
helm search repo nginx helm search hub wordpress
helm show values bitnami/postgresql helm show chart bitnami/postgresql
helm install my-release ./my-app \ --namespace production \ --create-namespace \ --values values-prod.yaml \ --set image.tag=v2.0.1 \ --set replicaCount=3
helm upgrade my-release ./my-app \ --namespace production \ --values values-prod.yaml \ --set image.tag=v2.0.2 \ --atomic \ --timeout 5m
helm rollback my-release 1 helm history my-release
helm template my-release ./my-app --values values-prod.yaml
helm install --dry-run --debug my-release ./my-app
helm uninstall my-release --namespace production
|
2.7 Helm Hooks
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
| apiVersion: batch/v1 kind: Job metadata: name: {{ include "my-app.fullname" . }}-migrate annotations: "helm.sh/hook": pre-install,pre-upgrade "helm.sh/hook-weight": "-5" "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded spec: template: spec: restartPolicy: Never containers: - name: migrate image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}" command: ["./migrate", "--up"]
|
3. Kustomize 深度解析
3.1 核心理念
Kustomize 使用**覆盖(Overlay)**而非模板:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
| base/(基础配置) ├── kustomization.yaml ├── deployment.yaml └── service.yaml
overlays/ ├── dev/(开发环境覆盖) │ ├── kustomization.yaml │ └── patch-replicas.yaml ├── staging/ │ ├── kustomization.yaml │ └── patch-resources.yaml └── prod/(生产环境覆盖) ├── kustomization.yaml ├── patch-replicas.yaml └── patch-resources.yaml
|
3.2 base 配置
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
| apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization
resources: - deployment.yaml - service.yaml - configmap.yaml
commonLabels: app: my-app managed-by: kustomize
images: - name: my-app newTag: latest
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
| apiVersion: apps/v1 kind: Deployment metadata: name: my-app spec: replicas: 1 selector: matchLabels: app: my-app template: spec: containers: - name: my-app image: my-app:latest resources: requests: cpu: 100m memory: 128Mi
|
3.3 生产环境 Overlay
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42
| apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization
namespace: production
resources: - ../../base
images: - name: my-app newName: my-registry/my-app newTag: v2.0.1
namePrefix: prod-
commonLabels: environment: production
patches: - path: patch-replicas.yaml - path: patch-resources.yaml
configMapGenerator: - name: app-config literals: - LOG_LEVEL=warn - APP_ENV=production envs: - prod.env
secretGenerator: - name: app-secret files: - secret.properties type: Opaque
|
1 2 3 4 5 6 7
| apiVersion: apps/v1 kind: Deployment metadata: name: my-app spec: replicas: 5
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
| apiVersion: apps/v1 kind: Deployment metadata: name: my-app spec: template: spec: containers: - name: my-app resources: requests: cpu: 500m memory: 512Mi limits: cpu: 2 memory: 2Gi
|
3.4 JSON Patch(精确修改)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
| patches: - target: kind: Deployment name: my-app patch: |- - op: replace path: /spec/replicas value: 5 - op: add path: /spec/template/spec/containers/0/env/- value: name: NEW_ENV value: "production" - op: remove path: /spec/template/spec/containers/0/env/0
|
3.5 Kustomize 常用命令
1 2 3 4 5 6 7 8 9 10 11
| kubectl kustomize overlays/prod
kubectl apply -k overlays/prod
kubectl delete -k overlays/prod
kubectl diff -k overlays/prod
|
4. Helm vs Kustomize 对比
| 特性 |
Helm |
Kustomize |
| 模板引擎 |
Go Template |
无(纯 YAML 覆盖) |
| 包管理 |
支持(Chart 仓库) |
不支持 |
| 版本管理 |
支持(Release 历史) |
依赖 Git |
| 学习曲线 |
较陡 |
平缓 |
| 灵活性 |
高 |
中 |
| 适用场景 |
复杂应用、第三方软件 |
内部应用多环境管理 |
| GitOps 集成 |
ArgoCD/Flux 支持 |
ArgoCD/Flux 原生支持 |
5. GitOps 工作流
5.1 ArgoCD 集成 Helm
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
| apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: my-app-prod namespace: argocd spec: project: default source: repoURL: https://github.com/my-org/my-app targetRevision: main path: helm/my-app helm: valueFiles: - values-prod.yaml parameters: - name: image.tag value: v2.0.1 destination: server: https://kubernetes.default.svc namespace: production syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true
|
5.2 ArgoCD 集成 Kustomize
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
| apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: my-app-prod namespace: argocd spec: source: repoURL: https://github.com/my-org/my-app targetRevision: main path: overlays/prod destination: server: https://kubernetes.default.svc namespace: production syncPolicy: automated: prune: true selfHeal: true
|
6. 最佳实践
1 2 3 4 5 6 7 8 9 10 11 12
| 1. 使用 --atomic 确保升级失败自动回滚 2. 在 CI/CD 中使用 helm lint 检查 Chart 3. 使用 helm test 运行集成测试 4. 将 values 文件存储在 Git 中(不含 Secret) 5. 使用 Sealed Secrets 或 External Secrets 管理敏感值
1. base 只包含最小化配置 2. 每个环境一个 overlay 目录 3. 使用 configMapGenerator 自动添加 hash 后缀(触发滚动更新) 4. 结合 ArgoCD/Flux 实现 GitOps
|
7. 总结
- Helm:适合打包和分发复杂应用,第三方软件首选
- Kustomize:适合内部应用的多环境配置管理,无需学习模板语言
- 实际项目:两者可以结合使用(Helm 管理第三方依赖,Kustomize 管理自研应用)
- GitOps:ArgoCD/Flux + Helm/Kustomize 是现代 K8s 应用交付的标准模式