无法从apache Spark连接到cassandra：com.datastax.oss.driver.api.core.connection.ClosedConnectionException：与远程对等点失去连接

Question

[从 docker windows 运行的 cassandra]

我正在运行来自 wsl2 的 Spark

spark-shell --packages com.datastax.spark:spark-cassandra-connector_2.12:3.5.1

[上面命令后的 Spark-shell]

当我尝试做的时候

data.collect().foreach(println)

我收到此错误

scala> data.collect().foreach(println)
24/11/21 23:57:24 WARN ControlConnection: [s0] Error connecting to Node(endPoint=localhost/127.0.0.1:9042, hostId=null, hashCode=62a6b9af), trying next node (ConnectionInitException: [s0|control|id: 0x29a613af, L:/127.0.0.1:43036 - R:localhost/127.0.0.1:9042] Protocol initialization request, step 1 (OPTIONS): unexpected failure (com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer))
java.io.IOException: Failed to open native connection to Cassandra at {localhost:9042} :: Could not reach any contact point, make sure you've provided valid addresses (showing first 1 nodes, use getAllErrors() for more): Node(endPoint=localhost/127.0.0.1:9042, hostId=null, hashCode=62a6b9af): [com.datastax.oss.driver.api.core.connection.ConnectionInitException: [s0|control|id: 0x29a613af, L:/127.0.0.1:43036 - R:localhost/127.0.0.1:9042] Protocol initialization request, step 1 (OPTIONS): unexpected failure (com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer)]
  at com.datastax.spark.connector.rdd.CassandraTableScanRDD.verify(CassandraTableScanRDD.scala:59)
  at com.datastax.spark.connector.rdd.CassandraTableScanRDD.getPartitions(CassandraTableScanRDD.scala:261)
  at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:294)
  at scala.Option.getOrElse(Option.scala:189)
  at org.apache.spark.rdd.RDD.partitions(RDD.scala:290)
  at org.apache.spark.SparkContext.runJob(SparkContext.scala:2458)
  at org.apache.spark.rdd.RDD.$anonfun$collect$1(RDD.scala:1049)
  at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
  at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
  at org.apache.spark.rdd.RDD.withScope(RDD.scala:410)
  at org.apache.spark.rdd.RDD.collect(RDD.scala:1048)
  ... 49 elided
Caused by: com.datastax.oss.driver.api.core.AllNodesFailedException: Could not reach any contact point, make sure you've provided valid addresses (showing first 1 nodes, use getAllErrors() for more): Node(endPoint=localhost/127.0.0.1:9042, hostId=null, hashCode=62a6b9af): [com.datastax.oss.driver.api.core.connection.ConnectionInitException: [s0|control|id: 0x29a613af, L:/127.0.0.1:43036 - R:localhost/127.0.0.1:9042] Protocol initialization request, step 1 (OPTIONS): unexpected failure (com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer)]
  at com.datastax.spark.connector.cql.CassandraConnector$.createSession(CassandraConnector.scala:167)
  ... 73 more
  Suppressed: com.datastax.oss.driver.api.core.connection.ConnectionInitException: [s0|control|id: 0x29a613af, L:/127.0.0.1:43036 - R:localhost/127.0.0.1:9042] Protocol initialization request, step 1 (OPTIONS): unexpected failure (com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer)
com.datastax.oss.driver.shaded.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
    at com.datastax.oss.driver.shaded.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at com.datastax.oss.driver.shaded.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.lang.Thread.run(Thread.java:750)
  Caused by: com.datastax.oss.driver.api.core.connection.ClosedConnectionException: Lost connection to remote peer

我检查了版本兼容性参考 https://github.com/datastax/spark-cassandra-connector 兼容性

我的版本 卡桑德拉：5.0.2 cqlsh：6.2.0（与 Cassandra 5.0.2 兼容） 火花：3.5.3 斯卡拉：2018 年 12 月 2 日 Java：1.8.0_432（Java 8）

对于cassandra连接器，我尝试了3.5.1和3.5.0，但都不起作用。 我也在 Windows 中尝试了相同的操作（也在 docker 上运行 Spark），然后问题仍然存在。

Answer 1

我相信您没有正确设置 Cassandra 连接，请将

127.0.0.1

替换为 cassandra 数据库主机的正确 IP 地址：

$SPARK_HOME/bin/spark-shell --conf spark.cassandra.connection.host=127.0.0.1 \
                            --packages com.datastax.spark:spark-cassandra-connector_2.12:3.5.1
                            --conf spark.sql.extensions=com.datastax.spark.connector.CassandraSparkExtensions