我一直在尝试在minikube上部署spark和jupyter note。我使用 helm charts 来部署两者
Jupyter 笔记本 - https://artifacthub.io/packages/helm/pyspark-notebook-helm/pyspark-notebook
Spark - https://bitnami.com/stack/spark/helm
虽然能够建立到主人使用
spark = SparkSession.builder.master("spark://my-release-spark-master-0.my-release-spark-headless.default.svc.cluster.local:7077").getOrCreate()
运行以下代码片段时
nums= sc.parallelize([1,2,3,4])
squared = nums.map(lambda x: x*x).collect()
for num in squared:
print('%i ' % (num))
运行collect()方法时执行耗时很长,永远不会完成