将 Env 值传递给 Helm 以创建 metricbeat

问题描述 投票：0回答：1

我在 Azure 中使用托管 ElasticCloud。我需要在 AKS 上使用 helm charts 安装 metricbeats。要配置这些节拍，我需要提供弹性云详细信息，例如 cloud.id 和 cloud.auth。我想从 github 环境传递这个值。

为此，我在 Github 环境中将这些变量添加为秘密。

这就是我的 values.yaml 的样子：

daemonset:
  annotations: {}
  labels: {}
  affinity: {}
  enabled: true
  envFrom: []
  extraEnvs: []
  extraVolumes: []
  extraVolumeMounts: []
  hostAliases: []
  hostNetworking: false
  
  metricbeatConfig:
    metricbeat.yml: |
      metricbeat.modules:
      - module: kubernetes
        metricsets:
          - container
          - node
          - pod
          - system
          - volume
        period: 10s
        host: "${NODE_NAME}"
        hosts: ["https://${NODE_NAME}:10250"]
        bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
        ssl.verification_mode: "none"
        
        processors:
        - add_kubernetes_metadata: ~
      - module: kubernetes
        enabled: true
        metricsets:
          - event
      - module: system
        period: 10s
        metricsets:
          - cpu
          - load
          - memory
          - network
          - process
          - process_summary
        processes: ['.*']
        process.include_top_n:
          by_cpu: 5
          by_memory: 5
      - module: system
        period: 1m
        metricsets:
          - filesystem
          - fsstat
        processors:
        - drop_event.when.regexp:
            system.filesystem.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib)($|/)'
      cloud.id: elk:ZWFzdHVzLmF6dXJlLmVsYXN0aWMtY2xvdWQuY29tOjQ0MyRmZjI5MjNjYTNkNTI0Yzg1ODBiMDk0ZGQxNTRmODQ5ZSQ3MmEyYjY5OTdjMWQ0ODFjYTViYjlkZTY3OGIxMzAzZA==
      cloud.auth: elastic:jixuoLFqs0vneFNaByIQdtL9
      output.elasticsearch:
        protocol: https
        ssl.verification_mode: "none"

  nodeSelector: {}
  
  securityContext:
    runAsUser: 0
    privileged: false
  resources:
    requests:
      cpu: "100m"
      memory: "100Mi"
    limits:
      cpu: "1000m"
      memory: "200Mi"
  tolerations: []

deployment:
  annotations: {}
  labels: {}
  affinity: {}
  enabled: true
  envFrom: []
  
  extraEnvs: []
  
  extraVolumes: []
 
  extraVolumeMounts: []
  
  hostAliases: []
  
  metricbeatConfig:
    metricbeat.yml: |
      metricbeat.modules:
      - module: kubernetes
        enabled: true
        metricsets:
          - state_node
          - state_deployment
          - state_replicaset
          - state_pod
          - state_container
        period: 10s
        hosts: ["${KUBE_STATE_METRICS_HOSTS}"]
      cloud.id: elk:ZWFzdHVzLmF6dXJlLmVsYXN0aWMtY2xvdWQuY29tOjQ0MyRmZjI5MjNjYTNkNTI0Yzg1ODBiMDk0ZGQxNTRmODQ5ZSQ3MmEyYjY5OTdjMWQ0ODFjYTViYjlkZTY3OGIxMzAzZA==
      cloud.auth: elastic:jixuoLFqs0vneFNaByIQdtL9
      output.elasticsearch:
        protocol: https
        ssl.verification_mode: "none"

  nodeSelector: {}
  secretMounts: []
  securityContext:
    runAsUser: 0
    privileged: false
  resources:
    requests:
      cpu: "100m"
      memory: "100Mi"
    limits:
      cpu: "1000m"
      memory: "200Mi"
  tolerations: []

replicas: 1

extraContainers: ""

extraInitContainers: ""

hostPathRoot: /var/lib

image: "docker.elastic.co/beats/metricbeat"
imageTag: "7.17.3"
imagePullPolicy: "IfNotPresent"
imagePullSecrets: []

livenessProbe:
  exec:
    command:
      - sh
      - -c
      - |
        #!/usr/bin/env bash -e
        curl --fail 127.0.0.1:5066
  failureThreshold: 3
  initialDelaySeconds: 10
  periodSeconds: 10
  timeoutSeconds: 5

readinessProbe:
  exec:
    command:
      - sh
      - -c
      - |
        #!/usr/bin/env bash -e
        metricbeat test output
  failureThreshold: 3
  initialDelaySeconds: 10
  periodSeconds: 10
  timeoutSeconds: 5

managedServiceAccount: true

clusterRoleRules:
  - apiGroups: [""]
    resources:
      - nodes
      - namespaces
      - events
      - pods
      - services
    verbs: ["get", "list", "watch"]
  - apiGroups: ["extensions"]
    resources:
      - replicasets
    verbs: ["get", "list", "watch"]
  - apiGroups: ["apps"]
    resources:
      - statefulsets
      - deployments
      - replicasets
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources:
      - nodes/stats
    verbs: ["get"]
  - apiGroups:
      - ""
    resources:
      - nodes/stats
    verbs:
      - get
  - nonResourceURLs:
      - "/metrics"
    verbs:
      - get

podAnnotations:
  {}
serviceAccount: ""

serviceAccountAnnotations:
  {}

terminationGracePeriod: 30

priorityClassName: ""

updateStrategy: RollingUpdate

nameOverride: ""
fullnameOverride: ""

kube_state_metrics:
  enabled: true
  host: ""

secrets: []

secret:
  name: elastic
data:
  cloud.id: elk:ZWFzdHVzLmF6dXJlLmVsYXN0aWMtY2xvdWQuY29tOjQ0MyRmZjI5MjNjYTNkNTI0Yzg1ODBiMDk0ZGQxNTRmODQ5ZSQ3MmEyYjY5OTdjMWQ0ODFjYTViYjlkZTY3OGIxMzAzZA==
  cloud.auth: elastic:jixuoLFqs0vneFNaByIQdtL9

在模板文件夹下- 秘密.yaml

apiVersion: v1
kind: Secret
metadata:
  name: {{ .Values.secret.name }}
type: Opaque
data:
  CLOUD_ID: {{ .Values.data.cloud.id }}
  CLOUD_AUTH: {{ .Values.data.cloud.auth }}

并将其作为 --

传递到 deployment.yaml 和 daemonset.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: '{{ template "metricbeat.fullname" . }}-metrics'
  labels:
    app: '{{ template "metricbeat.fullname" . }}-metrics'
    chart: '{{ .Chart.Name }}-{{ .Chart.Version }}'
    heritage: '{{ .Release.Service }}'
    release: '{{ .Release.Name }}'
    {{- if .Values.deployment.labels }}
    {{- range $key, $value := .Values.deployment.labels }}
    {{ $key }}: {{ $value | quote }}
    {{- end }}
    {{- else }}
    {{- range $key, $value := .Values.labels }}
    {{ $key }}: {{ $value | quote }}
    {{- end }}
    {{- end }}
  {{- if .Values.deployment.annotations}}
  annotations:
    {{- range $key, $value := .Values.deployment.annotations }}
    {{ $key }}: {{ $value | quote }}
    {{- end }}
  {{- end }}
spec:
  replicas: {{ .Values.replicas }}
  selector:
    matchLabels:
      app: '{{ template "metricbeat.fullname" . }}-metrics'
      release: '{{ .Release.Name }}'
  template:
    metadata:
      annotations:
        {{- range $key, $value := .Values.podAnnotations }}
        {{ $key }}: {{ $value | quote }}
        {{- end }}
        {{/* This forces a restart if the configmap has changed */}}
        {{- if or  .Values.metricbeatConfig .Values.deployment.metricbeatConfig }}
        configChecksum: {{ include (print .Template.BasePath "/configmap.yaml") . | sha256sum | trunc 63 }}
        {{- end }}
      labels:
        app: '{{ template "metricbeat.fullname" . }}-metrics'
        chart: '{{ .Chart.Name }}-{{ .Chart.Version }}'
        release: '{{ .Release.Name }}'
        {{- if .Values.deployment.labels }}
        {{- range $key, $value := .Values.deployment.labels }}
        {{ $key }}: {{ $value | quote }}
        {{- end }}
        {{- else }}
        {{- range $key, $value := .Values.labels }}
        {{ $key }}: {{ $value | quote }}
        {{- end }}
        {{- end }}
    spec:
      affinity: {{ toYaml .Values.deployment.affinity | nindent 8 }}
      nodeSelector: {{ toYaml .Values.deployment.nodeSelector | nindent 8 }}
      tolerations: {{ toYaml ( .Values.tolerations | default .Values.deployment.tolerations ) | nindent 8 }}
      {{- if .Values.priorityClassName }}
      priorityClassName: {{ .Values.priorityClassName  }}
      {{- end }}
      serviceAccountName: {{ template "metricbeat.serviceAccount" . }}
      terminationGracePeriodSeconds: {{ .Values.terminationGracePeriod }}
      {{- if .Values.deployment.hostAliases }}
      hostAliases: {{ toYaml .Values.deployment.hostAliases | nindent 6 }}
      {{- end }}
      volumes:
      {{- range .Values.secretMounts | default .Values.deployment.secretMounts }}
      - name: {{ .name }}
        secret:
          secretName: {{ .secretName }}
      {{- end }}
      {{- if .Values.metricbeatConfig }}
      - name: metricbeat-config
        configMap:
          defaultMode: 0600
          name: {{ template "metricbeat.fullname" . }}-config
      {{- else if .Values.deployment.metricbeatConfig }}
      - name: metricbeat-config
        configMap:
          defaultMode: 0600
          name: {{ template "metricbeat.fullname" . }}-deployment-config
      {{- end }}
      {{- if .Values.extraVolumes | default .Values.deployment.extraVolumes }}
{{ toYaml ( .Values.extraVolumes | default .Values.deployment.extraVolumes ) | indent 6 }}
      {{- end }}
      {{- if .Values.imagePullSecrets }}
      imagePullSecrets:
{{ toYaml .Values.imagePullSecrets | indent 8 }}
      {{- end }}
      {{- if .Values.extraInitContainers }}
      initContainers:
{{ tpl .Values.extraInitContainers . | indent 6 }}
      {{- end }}
      containers:
      - name: "metricbeat"
        image: "{{ .Values.image }}:{{ .Values.imageTag }}"
        imagePullPolicy: "{{ .Values.imagePullPolicy }}"
        args:
        {{- if index .Values "metricbeatConfig" "kube-state-metrics-metricbeat.yml" }}
          - "-c"
          - "/usr/share/metricbeat/kube-state-metrics-metricbeat.yml"
        {{- end }}
          - "-e"
          - "-E"
          - "http.enabled=true"
        livenessProbe:
{{ toYaml .Values.livenessProbe | indent 10 }}
        readinessProbe:
{{ toYaml .Values.readinessProbe | indent 10 }}
        resources: {{ toYaml ( .Values.resources | default .Values.deployment.resources ) | nindent 10 }}
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: KUBE_STATE_METRICS_HOSTS
          {{- if .Values.kube_state_metrics.enabled }}
          value: "$({{ .Release.Name | replace "-" "_" | upper }}_KUBE_STATE_METRICS_SERVICE_HOST):$({{ .Release.Name | replace "-" "_" | upper }}_KUBE_STATE_METRICS_SERVICE_PORT_HTTP)"
          {{- else }}
          value: {{ .Values.kube_state_metrics.host | default "kube-state-metrics:8080"}}
          {{- end }}
        - name: cloud.id
          valueFrom:
            secretKeyRef:
              name: {{ .Values.secret.name }}
              key: CLOUD_ID
        - name: cloud.auth
          valueFrom:
            secretKeyRef:
              name: {{ .Values.secret.name }}
              key: CLOUD_AUTH

{{- if .Values.extraEnvs | default .Values.deployment.extraEnvs }}
{{ toYaml ( .Values.extraEnvs | default .Values.deployment.extraEnvs ) | indent 8 }}
{{- end }}
        envFrom: {{ toYaml ( .Values.envFrom | default .Values.deployment.envFrom ) | nindent 10 }}
        securityContext: {{ toYaml ( .Values.podSecurityContext | default .Values.deployment.securityContext ) | nindent 10 }}
        volumeMounts:
        {{- range .Values.secretMounts | default .Values.deployment.secretMounts }}
        - name: {{ .name }}
          mountPath: {{ .path }}
          {{- if .subPath }}
          subPath: {{ .subPath }}
          {{- end }}
        {{- end }}
        {{- range $path, $config := .Values.metricbeatConfig }}
        - name: metricbeat-config
          mountPath: /usr/share/metricbeat/{{ $path }}
          readOnly: true
          subPath: {{ $path }}
        {{ else }}
        {{- range $path, $config := .Values.deployment.metricbeatConfig }}
        - name: metricbeat-config
          mountPath: /usr/share/metricbeat/{{ $path }}
          readOnly: true
          subPath: {{ $path }}
        {{- end }}
        {{- end }}
        {{- if .Values.extraVolumeMounts | default .Values.deployment.extraVolumeMounts }}
{{ toYaml ( .Values.extraVolumeMounts | default .Values.deployment.extraVolumeMounts ) | indent 8 }}
        {{- end }}
      {{- if .Values.extraContainers }}
{{ tpl .Values.extraContainers . | indent 6 }}
      {{- end }}
{{- end }}

这就是我执行 helm 的方式：

helm upgrade --install mbeat elastic/metricbeat --version 7.17.3 -f ./values.yaml --set cloud.id=${{ secrets.CLOUD_ID }},cloud.auth=${{ secrets.CLOUD_AUTH }}

但它仍然采用 values.yaml 中给出的 cloud.id 和 cloud.auth 的值，它没有使用我通过 helm upgrade install 命令设置的值

kubernetes-helm elastic-stack helm3 metricbeat
1个回答
0
投票

你生成的秘密文件会是这样的

模板/secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: {{ .Values.secret.name }}
data:
  CLOUD_ID: {{ .Values.data.cloud_id }}

这是你的values.yaml

replicaCount: 1

secret:
  name: common

data:
  cloud_id: "xyz"
  username: "user"
  password: "admin123"

所以来自 values.yaml 的值被应用于秘密并创建了模板。

问题中提到的错误：

Exiting: error loading config file: yaml: invalid map key: map[interface {}]interface {}{".Values.cloud.id":interface {}(nil)}

我无法找到您使用的引发错误的

.Values.cloud.id
。

values.yaml

daemonset:
  metricbeatConfig:
    metricbeat.yml:
      metricbeat.modules:
        - module: kubernetes
          enabled: true
          metricsets:
            - state_node
            - state_deployment
            - state_replicaset
            - state_pod
            - state_container
          period: 10s
          hosts: ["${KUBE_STATE_METRICS_HOSTS}"]
        cloud.id: cloudId

secret.yaml

apiVersion: v1
kind: secret
metadata:
  name: secret-name
data:
{{- $root := . -}}
{{- range $key, $value := .Values.daemonset.metricbeatConfig }}
  {{- if $key | regexMatch ".*\\.ya?ml$" }}
  {{ $key }}: |
{{ toYaml $value | default "{}" | indent 4 }}
  {{- else }}
  {{ $key }}: {{ toYaml $value | indent 4 }}
  {{- end }}
{{- end -}}

它会创造秘密之类的东西

apiVersion: v1
kind: secret
metadata:
  name: secret-name
data:
  metricbeat.yml: |
      metricbeat.modules:
        - module: kubernetes
          enabled: true
          metricsets:
            - state_node
            - state_deployment
            - state_replicaset
            - state_pod
            - state_container
          period: 10s
          hosts: ["${KUBE_STATE_METRICS_HOSTS}"]
        cloud.id: cloudId

如果您尝试从 values.yaml 获取 cloud.id 的value，您不能直接获取它，因为它是 metricbeat.yml 文件的一部分。如果您正在寻找那个，请告诉我。

最新问题
© www.soinside.com 2019 - 2024. All rights reserved.