我正在尝试本地Kubernetes(Mac上的Docker),并尝试提交Spark作业。 Spark作业与PostgreSQL数据库连接并进行一些计算。
PostgreSQL在我的Kube上运行,并且由于我已经发布了它,所以我可以通过localhost:5432从主机访问它。但是,当spark应用程序尝试连接到PostgreSQL时,它将引发
Exception in thread "main" org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
kubectl群集信息
Kubernetes master is running at https://kubernetes.docker.internal:6443
KubeDNS is running at https://kubernetes.docker.internal:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
kubectl获取服务postgresql发布
kubectl描述服务spark-store-1588217023181-driver-svc
Name: spark-store-1588217023181-driver-svc
Namespace: default
Labels: <none>
Annotations: <none>
Selector: spark-app-selector=spark-533ecb8556b6439eb938d487cc77c330,spark-role=driver
Type: ClusterIP
IP: None
Port: driver-rpc-port 7078/TCP
TargetPort: 7078/TCP
Endpoints: <none>
Port: blockmanager 7079/TCP
TargetPort: 7079/TCP
Endpoints: <none>
Session Affinity: None
我如何使我的火花工作,可以访问PostgreSQL服务?
Spark Pod无法通过本地主机访问Postgres,因为Postgres作为单独的Pod运行。应通过10.106.15.112:5432
或postgresql-published:5432
从Spark Pod访问Postgres。