h2o.xgboost火车错误java.lang.NullPointerException h2o版本:3.26.0.2

问题描述 投票:0回答:1

我目前正在使用h2o 3.26.0.2版中的h2o.xgboost,并且得到了java.lang.NullPointerException(下面有完整错误)。

[数据集在csv中为25 GB,具有6.000.000行(trian +测试),我使用的群集信息为:

R is connected to the H2O cluster: 
    H2O cluster uptime:         30 minutes 18 seconds 
    H2O cluster timezone:       Europe/Madrid 
    H2O data parsing timezone:  UTC 
    H2O cluster version:        3.26.0.2 
    H2O cluster version age:    7 months and 12 days !!! 
    H2O cluster name:           H2O_started_from_R_xxx
    H2O cluster total nodes:    1 
    H2O cluster total memory:   343.27 GB 
    H2O cluster total cores:    1 
    H2O cluster allowed cores:  48 
    H2O cluster healthy:        TRUE 
    H2O Connection ip:          localhost 
    H2O Connection port:        60576 
    H2O Connection proxy:       NA 
    H2O Internal Security:      FALSE 
    H2O API Extensions:         Amazon S3, XGBoost, Algos, AutoML, Core V3, Core V4 
    R Version:                  R version 3.5.1 (2018-07-02) 

数据h2o类型为:

 types   N
1:   int 319
2:  real 316

和我的代码:

start <- proc.time()
model_trans.h2o <- h2o::h2o.xgboost( 
  distribution              = "gaussian"
  ,model_id                 = "xgb_test"
  ,training_frame           = data_train.h2o
  ,validation_frame         = data_test.h2o
  ,x                        = vars
  ,y                        = target_str
  ,seed                     = 1234
  ,ntrees                   = 1500
  ,learn_rate               = 0.08
  ,col_sample_rate_per_tree = 0.8
  ,max_dept                 = 5
  ,verbose                  = F
)
end <- proc.time(); end - start;

通常在2分钟后给出错误,并且我进行了以下测试:

  1. 使用较少的列。使用500列工作正常(我没有使用来自同名的1:500的随机样本)。
  2. 使用较少的数据。如果我减少数据,那么也说2.000.000行,它工作正常

[在测试之后,我猜测正在发生的事情是h2o.xgboost不准备处理这么多数据,并且当尝试扩展它时,“矩阵”正在崩溃。

可能是H2O版本的错误吗?

可以在不更新版本的情况下解决吗?由于h2o模型与跨版本不兼容,因此这对我来说可能是个问题。

提前感谢!

注1:

我已经完成了对该主题的先前研究,但仅发现:

What is a NullPointerException, and how do I fix it? SO讨论在空指针异常之上

[https://0xdata.atlassian.net/browse/PUBDEV-6921一些h2o论坛没有明确答案(第二个几乎没有用)

注2:完整错误是:

java.lang.NullPointerException

java.lang.NullPointerException
    at hex.tree.xgboost.matrix.SparseMatrixFactory$NestedArrayPointer.set(SparseMatrixFactory.java:87)
    at hex.tree.xgboost.matrix.SparseMatrixFactory$InitializeCSRMatrixFromChunkIdsMrFun.map(SparseMatrixFactory.java:178)
    at water.LocalMR.compute2(LocalMR.java:84)
    at water.LocalMR.compute2(LocalMR.java:76)
    at water.LocalMR.compute2(LocalMR.java:76)
    at water.LocalMR.compute2(LocalMR.java:76)
    at water.LocalMR.compute2(LocalMR.java:76)
    at water.LocalMR.compute2(LocalMR.java:76)
    at water.LocalMR.compute2(LocalMR.java:76)
    at water.LocalMR.compute2(LocalMR.java:76)
    at water.H2O$H2OCountedCompleter.compute(H2O.java:1417)
    at jsr166y.CountedCompleter.exec(CountedCompleter.java:468)
    at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
    at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
    at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
    at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
java nullpointerexception xgboost h2o
1个回答
© www.soinside.com 2019 - 2024. All rights reserved.