获取访问令牌以在 aws eks 作业提交中使用，返回错误 java.lang.IllegalArgumentException:

Question

从 EC2 运行 spark-submit 命令我为用户生成一个令牌

https://awscli.amazonaws.com/v2/documentation/api/latest/reference/eks/get-token.html

aws eks get-token --cluster-name spark-on-eks --region eu-west-2 > token

这回来了一个令牌

{
    "kind": "ExecCredential",
    "apiVersion": "client.authentication.k8s.io/v1beta1",
    "spec": {},
    "status": {
        "expirationTimestamp": "2023-03-30T10:02:17Z",
        "token": "k8s-aws-v1.aHR0cHM6Ly9zdHMuZXUtd2VzdC0yLmFtYXpvbmF3cy5jb20vP0FjdGlvbj1HZXRDYWxsZXJJZGVudGl0eSZWZXJzaW9uPTIwMTEtMDYtMTUmWC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBUldRUzZVQ0JKTjdWQjZFQiUyRjIwMjMwMzMwJTJGZXUtd2VzdC0yJTJGc3RzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyMzAzMzBUMDk0ODE3WiZYLUFtei1FeHBpcmVzPTYwJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCUzQngtazhzLWF3cy1pZCZYLUFtei1TaWduYXR1cmU9NGUxZGE5MzkxYjZmMGFhM2M3ZDQyM2EyMGQyNTRhMWU4NWQ4M2IwNmEzNTRiNmExOWRkMWIxNmMzODZiYTA0YQ"
    }
}

在 spark-submit 中，我将其传递为

export K8S_TOKEN=/var/run/secrets/kubernetes.io/serviceaccount/token
--conf spark.kubernetes.authenticate.submission.oauthTokenFile=$K8S_TOKEN 

但是它抛出这个错误

Exception in thread "main" java.lang.IllegalArgumentException: Unexpected char 0x0a at 8 in Authorization value: Bearer {
    "kind": "ExecCredential",
    "apiVersion": "client.authentication.k8s.io/v1beta1",
    "spec": {},
    "status": {
        "expirationTimestamp": "2023-03-30T10:02:17Z",
        "token": "k8s-aws-v1.aHR0cHM6Ly9zdHMuZXUtd2VzdC0yLmFtYXpvbmF3cy5jb20vP0FjdGlvbj1HZXRDYWxsZXJJZGVudGl0eSZWZXJzaW9uPTIwMTEtMDYtMTUmWC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBUldRUzZVQ0JKTjdWQjZFQiUyRjIwMjMwMzMwJTJGZXUtd2VzdC0yJTJGc3RzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyMzAzMzBUMDk0ODE3WiZYLUFtei1FeHBpcmVzPTYwJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCUzQngtazhzLWF3cy1pZCZYLUFtei1TaWduYXR1cmU9NGUxZGE5MzkxYjZmMGFhM2M3ZDQyM2EyMGQyNTRhMWU4NWQ4M2IwNmEzNTRiNmExOWRkMWIxNmMzODZiYTA0YQ"
    }
}


        at okhttp3.Headers.checkValue(Headers.java:272)
        at okhttp3.Headers$Builder.add(Headers.java:312)
        at okhttp3.Request$Builder.addHeader(Request.java:196)
        at io.fabric8.kubernetes.client.utils.HttpClientUtils.lambda$createHttpClient$3(HttpClientUtils.java:150)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
        at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:257)
        at okhttp3.RealCall.execute(RealCall.java:93)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:490)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:451)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleCreate(OperationSupport.java:252)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleCreate(BaseOperation.java:879)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:341)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:84)
        at org.apache.spark.deploy.k8s.submit.Client.run(KubernetesClientApplication.scala:139)
        at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.$anonfun$run$3(KubernetesClientApplication.scala:213)
        at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.$anonfun$run$3$adapted(KubernetesClientApplication.scala:207)
        at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:2611)
        at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.run(KubernetesClientApplication.scala:207)
        at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.start(KubernetesClientApplication.scala:179)
        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951)
        at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
        at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1030)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1039)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
23/03

在这种情况下这是一个无效的令牌还是生成的令牌语法有问题？

谢谢

Answer 1

解决方案是从生成的令牌中去掉换行符

aws eks get-token --cluster-name $CLUSTER_NAME --region $REGION --output text | base64| tr -d '\n' > token