Spark 3.5.0 java.lang.NoSuchFieldError：JAVA_11

Question

尝试升级我的项目以使用 Spark 3.5.0 Maven 全新安装时遇到问题 我所有涉及 Spark 的测试都失败并出现以下错误

An exception or error caused a run to abort: JAVA_11 
java.lang.NoSuchFieldError: JAVA_11
    at org.apache.spark.internal.config.UI$.<init>(UI.scala:105)
    at org.apache.spark.internal.config.UI$.<clinit>(UI.scala)
    at org.apache.spark.SecurityManager.<init>(SecurityManager.scala:59)
    at org.apache.spark.SparkEnv$.create(SparkEnv.scala:262)
    at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:196)
    at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:284)
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:483)
    at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2888)
    at org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:1099)
    at scala.Option.getOrElse(Option.scala:189)
    at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:1093)
    at com.intuit.mlp.valkyrie.spark.ABCTest.<init>(ABCTest.scala:16)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at java.lang.Class.newInstance(Class.java:442)
    at org.scalatest.tools.Runner$.genSuiteConfig(Runner.scala:1428)
    at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$8(Runner.scala:1236)
    at scala.collection.immutable.List.map(List.scala:293)
    at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1235)
    at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24(Runner.scala:1031)
    at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24$adapted(Runner.scala:1010)
    at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1506)
    at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:1010)
    at org.scalatest.tools.Runner$.run(Runner.scala:850)
    at org.scalatest.tools.Runner.run(Runner.scala)
    at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.runScalaTest2or3(ScalaTestRunner.java:43)
    at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.main(ScalaTestRunner.java:26)

请建议一些方法来调试此问题 我正在使用 java 1.8_421 和 scala 2.12.12

尝试排除 commons-lang3 冲突，但没有成功。 尝试在 java 11 上运行应用程序但不起作用。 需要修复缺失的 JAVA_11 错误

Answer 1

我可以通过将 commons-lang3 更新到上面建议的最新版本来解决这个问题

      <dependency>
        <groupId>org.apache.commons</groupId>
        <artifactId>commons-lang3</artifactId>
        <version>3.17.0</version>
        <scope>provided</scope>
      </dependency>