启动 Optuna 时 CatBoost 崩溃

Question

我想使用 Optuna 和本地 GPU 调整回归器 catboost。数据集不是很大：训练样本包含大约 120k 条记录，只有 16 个特征（包括分类特征）。我运行代码：

RANDOM_SEED = 2906
n_trials = 100

def objective_cat(trial):
  params = {'iterations': trial.suggest_int('iterations', 100, 2000),
            'learning_rate': trial.suggest_float('learning_rate', 1e-5, 1e-1, log=True),
            'depth': trial.suggest_int('max_depth', 2, 16),
            'bagging_temperature': trial.suggest_uniform('bagging_temperature', 0, 1),
            'l2_leaf_reg': trial.suggest_float('l2_leaf_reg', 1e-5, 100, log=True),
            'random_seed': RANDOM_SEED,
            'loss_function': 'RMSE',
            'task_type': 'GPU'
            }

  cat_model = CatBoostRegressor(**params, cat_features=cat_features)
  cv = KFold(
      n_splits=10,
      shuffle=True,
      random_state=RANDOM_SEED
      )
  score = cross_val_score(
      cat_model, X_train, y_train, cv=cv, n_jobs=-1,
      scoring='neg_root_mean_squared_error'
      )
  return -np.mean(score)

study_cat = optuna.create_study(direction='minimize',
                                 sampler=optuna.samplers.TPESampler(
                                     seed=RANDOM_SEED,
                                     n_startup_trials=20,
                                     multivariate=True
                                     ),
                                 pruner=optuna.pruners.HyperbandPruner(
                                     min_resource=1,
                                     max_resource=100,
                                     reduction_factor=3
                                     ),
                                 study_name=f'optuna_cat')
optuna.logging.set_verbosity(optuna.logging.CRITICAL)
study_cat.optimize(
    objective_cat,
    n_trials=n_trials,
    n_jobs=-1,
    show_progress_bar=True
    )
best_params_cat = study_cat.best_params

出现错误：

TerminatedWorkerError                     Traceback (most recent call last)
Cell In[70], line 39
     26 study_cat = optuna.create_study(direction='minimize',
     27                                  sampler=optuna.samplers.TPESampler(
     28                                      seed=RANDOM_SEED,
   (...)
     36                                      ),
     37                                  study_name=f'optuna_cat')
     38 optuna.logging.set_verbosity(optuna.logging.CRITICAL)
---> 39 study_cat.optimize(
     40     objective_cat,
     41     n_trials=n_trials,
     42     n_jobs=-1,
     43     show_progress_bar=True
...
    764     return self._result
    765 finally:

TerminatedWorkerError: A worker process managed by the executor was unexpectedly terminated. This could be caused by a segmentation fault while calling the function or by an excessive memory usage causing the Operating System to kill the worker.

“原始”CatBoost（不含 Optuna 等）在我的 GPU 上运行良好。我该如何解决这个问题？

版本： 猫Boost 1.2.5 奥图纳 4.0.0 我的 GPU 是 NVIDIA GeForce RTX 3060

我尝试了使用 CPU 的 Optuna，但整整一个小时都没有迭代！

几乎尝试了滴

n_jobs=-1

并且：

os.environ['CUDA_VISIBLE_DEVICES'] = '0'

Answer 1

我从 KFolds 参数中删除了 n_jobs=-1 并且它有效。但很慢:-(