我目前正在使用h2o 3.26.0.2版中的h2o.xgboost,并且得到了java.lang.NullPointerException(下面有完整错误)。
[数据集在csv中为25 GB,具有6.000.000行(trian +测试),我使用的群集信息为:
R is connected to the H2O cluster:
H2O cluster uptime: 30 minutes 18 seconds
H2O cluster timezone: Europe/Madrid
H2O data parsing timezone: UTC
H2O cluster version: 3.26.0.2
H2O cluster version age: 7 months and 12 days !!!
H2O cluster name: H2O_started_from_R_xxx
H2O cluster total nodes: 1
H2O cluster total memory: 343.27 GB
H2O cluster total cores: 1
H2O cluster allowed cores: 48
H2O cluster healthy: TRUE
H2O Connection ip: localhost
H2O Connection port: 60576
H2O Connection proxy: NA
H2O Internal Security: FALSE
H2O API Extensions: Amazon S3, XGBoost, Algos, AutoML, Core V3, Core V4
R Version: R version 3.5.1 (2018-07-02)
数据h2o类型为:
types N
1: int 319
2: real 316
和我的代码:
start <- proc.time()
model_trans.h2o <- h2o::h2o.xgboost(
distribution = "gaussian"
,model_id = "xgb_test"
,training_frame = data_train.h2o
,validation_frame = data_test.h2o
,x = vars
,y = target_str
,seed = 1234
,ntrees = 1500
,learn_rate = 0.08
,col_sample_rate_per_tree = 0.8
,max_dept = 5
,verbose = F
)
end <- proc.time(); end - start;
通常在2分钟后给出错误,并且我进行了以下测试:
[在测试之后,我猜测正在发生的事情是h2o.xgboost不准备处理这么多数据,并且当尝试扩展它时,“矩阵”正在崩溃。
可能是H2O版本的错误吗?
可以在不更新版本的情况下解决吗?由于h2o模型与跨版本不兼容,因此这对我来说可能是个问题。
提前感谢!
注1:
我已经完成了对该主题的先前研究,但仅发现:
What is a NullPointerException, and how do I fix it? SO讨论在空指针异常之上
[https://0xdata.atlassian.net/browse/PUBDEV-6921一些h2o论坛没有明确答案(第二个几乎没有用)
注2:完整错误是:
java.lang.NullPointerException
java.lang.NullPointerException
at hex.tree.xgboost.matrix.SparseMatrixFactory$NestedArrayPointer.set(SparseMatrixFactory.java:87)
at hex.tree.xgboost.matrix.SparseMatrixFactory$InitializeCSRMatrixFromChunkIdsMrFun.map(SparseMatrixFactory.java:178)
at water.LocalMR.compute2(LocalMR.java:84)
at water.LocalMR.compute2(LocalMR.java:76)
at water.LocalMR.compute2(LocalMR.java:76)
at water.LocalMR.compute2(LocalMR.java:76)
at water.LocalMR.compute2(LocalMR.java:76)
at water.LocalMR.compute2(LocalMR.java:76)
at water.LocalMR.compute2(LocalMR.java:76)
at water.LocalMR.compute2(LocalMR.java:76)
at water.H2O$H2OCountedCompleter.compute(H2O.java:1417)
at jsr166y.CountedCompleter.exec(CountedCompleter.java:468)
at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)