Python Kubernetes MutatingWebhook 不添加标签

Question

我使用从 Python 构建的 Docker 映像创建了 Kubernetes MutatingWebhook。 Webhook 的目的是向任何启动的 Pod 添加标签。 所有 Kubernetes 资源都运行良好，但是，当启动新 Pod 时，它们无法从 MutatingWebhook 获得预期的标签。

启动新 Pod 时出现此错误：

Error from server (InternalError): Internal error occurred: failed calling webhook "add-labels.k8s.io": failed to call webhook: Post "https://add-labels-webhook-svc.mutatingwh.svc:443/mutate?timeout=10s": dial tcp 10.3.33.250:443: connect: connection refused

这是 Python webhook 代码：

import json
from flask import Flask, request, jsonify
import base64

app = Flask(__name__)

@app.route('/mutate', methods=['POST'])
def mutate():
    # Read the admission review request
    admission_review = request.get_json()
    print("AdmissionReview Request:", json.dumps(admission_review))

    # Define the patch to add labels
    patch = [
        {
            "op": "add",
            "path": "/metadata/labels/live",
            "value": "true"
        },
        {
            "op": "add",
            "path": "/metadata/labels/environment",
            "value": "production"
        },
        {
            "op": "add",
            "path": "/metadata/labels/service",
            "value": "my-service"
        },
        {
            "op": "add",
            "path": "/metadata/labels/version",
            "value": "7.55.2"
        }
    ]

    # Encode the patch to base64
    patch_base64 = base64.b64encode(json.dumps(patch).encode()).decode()

    # Prepare the admission review response
    admission_response = {
        "uid": admission_review['request']['uid'],
        "allowed": True,
        "patchType": "JSONPatch",
        "patch": patch_base64
    }

    # Return the admission review response
    response = {
        "apiVersion": "admission.k8s.io/v1",
        "kind": "AdmissionReview",
        "response": admission_response
    }

    return jsonify(response)

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=443, ssl_context=('/tls/tls.crt', '/tls/tls.key'), debug=True)

这是我用于部署、服务等的各种 K8s 清单文件

部署.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: add-labels-webhook
  namespace: mutatingwh
spec:
  replicas: 1
  selector:
    matchLabels:
      app: add-labels-webhook
  template:
    metadata:
      labels:
        app: add-labels-webhook
    spec:
      containers:
        - name: webhook
          image: 805960120419.dkr.ecr.us-east-1.amazonaws.com/noc/add-labels-webhook:latest
          ports:
            - containerPort: 443
          volumeMounts:
            - name: tls-certs
              mountPath: /tls
              readOnly: true
      imagePullSecrets:
        - name: ecr-secret
      volumes:
        - name: tls-certs
          secret:
            secretName: webhook-tls

服务.yaml

apiVersion: v1
kind: Service
metadata:
  name: add-labels-webhook-svc
  namespace: mutatingwh
spec:
  ports:
    - port: 443
      targetPort: 443
  selector:
    app: add-labels-webhook

mutatingwebhookconfiguration.yaml

apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
  name: add-labels-webhook
webhooks:
  - name: add-labels.k8s.io
    clientConfig:
      service:
        name: add-labels-webhook-svc
        namespace: mutatingwh
        path: "/mutate"
      caBundle: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURNekNDQWh1Z0F3SUJBZ0lVTEpJVlFkcXBiR05yb3N3Rk9SRDA0SzREbnpjd0RRWUpLb1pJaHZjTkFRRUwKQlFBd0tURW5NQ1VHQTFVRUF3d2VZV1JrTFd4aFltVnNjeTEzWldKb2IyOXJMbVJsWm1GMWJIUXVjM1pqTUI0WApEVEkwTVRBeU5UQTRORGd3T0ZvWERUSTFNVEF5TlRBNE5EZ3dPRm93S1RFbk1DVUdBMVVFQXd3ZVlXUmtMV3hoClltVnNjeTEzWldKb2IyOXJMbVJsWm1GMWJIUXVjM1pqTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEEKTUlJQkNnS0NBUUVBc05CWG0yWE1TZVNTUUxsZ2ZvR01wRk0xeHRtdmxxN1JrK2RoTUw0b211VVN1NDJaMk9VcAp1S3RJdkdtRnorYUpBU3FONmFTY0xJbFFTTElSdmNqNnJ3aXVwcGZVQldSNkdDeEhGYW52R0VxT0YxSkR6OEptCko4Zkp3b09aMFJqQmtYdWFLSFdtNC9NSHV6T240SUdnV3dRRnN2Tk1ZUitFMkRsRjZnOWE4c0cwbjVQTVVydHkKNGdiVDduY2pFZnc2NFc2SmZqRzFNTDNLTXI3aUZ1NXBxM0hJMXh2cSt5T2lDNGQ4WEVJL3FibU9vbDdOYUZDRwpyNVZRd0hLMWx4aHBZY2VWVTNlQWIxS0tHL3FDSVdURVptbTlFbGZuMWJRbzZtOEVxenVES2QyTHE1VmRqUWVnCmFJRkNMYmd2YWx0S1FWaGIwUEhNRjlqUE5yN0xLenRsclFJREFRQUJvMU13VVRBZEJnTlZIUTRFRmdRVUpYcDEKdnVmcXVFbW9VTDhmL3BUZXEzQ25vME13SHdZRFZSMGpCQmd3Rm9BVUpYcDF2dWZxdUVtb1VMOGYvcFRlcTNDbgpvME13RHdZRFZSMFRBUUgvQkFVd0F3RUIvekFOQmdrcWhraUc5dzBCQVFzRkFBT0NBUUVBSTNUczh0UE1zR29kCjZvU2ZxanQ0Rzk0SWR4SjVRZkZMaG5hTnBmS0tBRldDZVI2cVEzYWtWWFNnVzVFSlBud0dtdjBYRkh2YWtXa1EKc3A5QjhvaVI1OWlWRHpFME5QdytHQmkzMWkrTW9aaDAzYXYyWFg0K1FKOTFkRENldkFZeEQ4RkxackZ3UTJEMwpjUVdmQk1kSS9JSys5V1dWNXRtZmdtN2d3bi9XNy9nTC9ScVNkaVQwWllmRjZSVjhNeHFaano4bGFWUXhCU2FYCi9kNVROZi9XcldIUjlIek1xRU91UXBIL3NqRVlUM2xkbTA2aTdKMFVsOFJDQTkyL3c2L3dLRk82SHFYVWR1RVoKS25KTHdZV1A4YnJkMHhTcFhYUFRrbHl0T3ZLYkt3N2ljeDRaYzkxbXJOMjRGZ2k5OWN3L2tIZng4cnQ2RVJqawp0S3BLSCt2OVpRPT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
    admissionReviewVersions: ["v1"]
    sideEffects: None
    rules:
      - operations: ["CREATE"]
        apiGroups: [""]
        apiVersions: ["v1"]
        resources: ["pods"]
    failurePolicy: Fail

我的问题是为什么一切都运行良好，但是当创建新的 Pod 时，它们没有从 Python webhook 获取标签？

Answer 1

Kubernetes 准入 Webhook 要求 Webhook 服务器使用 HTTPS。我通过使用以下五个 openssl 命令生成 TLS 证书和密钥解决了我的问题：

为 CA 生成私钥

openssl genrsa -out ca.key 2048

为 CA 创建自签名证书

openssl req -x509 -new -nodes -key ca.key -subj "/CN=MutatingWebhookCA" -days 365 -out ca.crt

生成 Webhook 服务器私钥

openssl genrsa -out webhook-server.key 2048

为 Webhook 服务器创建证书签名请求 (CSR)

openssl req -new -key webhook-server.key -subj "/CN=add-labels-webhook-svc.mutatingwh.svc" -out webhook-server.csr

使用 CA 签署 Webhook 服务器证书

openssl x509 -req -in webhook-server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out webhook-server.crt -days 365 -extfile <(echo "subjectAltName=DNS:add-labels-webhook-svc.mutatingwh.svc")