XGBoost 提前停止回合

Question

我的代码（如下）不断崩溃，我不知道发生了什么，也不知道为什么需要任何帮助。已经通过各种LLM但无济于事。

import optuna
import xgboost as xgb
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error

# Assuming `X` and `y` are your feature matrix and target array
X_train, X_valid, y_train, y_valid = train_test_split(df_combined, y, test_size=0.2, random_state=42)

# Define the objective function for Optuna
def objective(trial):
    # Suggest values for hyperparameters
    params = {
        "objective": "reg:squarederror",
        "eval_metric": "rmse",
        "tree_method": "hist",  # Use hist method
        "device": "cuda",       # Specify using GPU
        "learning_rate": trial.suggest_float("learning_rate", 0.01, 0.3, log=True),
        "max_depth": trial.suggest_int("max_depth", 3, 10),
        "min_child_weight": trial.suggest_float("min_child_weight", 1, 10),
        "gamma": trial.suggest_float("gamma", 0, 1),
        "subsample": trial.suggest_float("subsample", 0.5, 1.0),
        "colsample_bytree": trial.suggest_float("colsample_bytree", 0.5, 1.0),
        "lambda": trial.suggest_float("lambda", 1e-3, 10.0, log=True),
        "alpha": trial.suggest_float("alpha", 1e-3, 10.0, log=True),
        "n_estimators": 1000  # Define n_estimators in the initialization of the model
    }

    # Initialize the model
    model = xgb.XGBRegressor(**params)

    # Train the model with early stopping callback
    model.fit(
        X_train,
        y_train,
        eval_set=[(X_valid, y_valid)],
        verbose=False,
        early_stopping_rounds=50  # Stops if no improvement after 50 rounds
    )

    # Predict and calculate RMSE for validation set
    preds = model.predict(X_valid)
    rmse = mean_squared_error(y_valid, preds, squared=False)

    return rmse  # Optuna minimizes this

# Set up the Optuna study
study = optuna.create_study(direction="minimize")

# Optimize the hyperparameters
study.optimize(objective, n_trials=100, n_jobs=40)  # 100 trials with 40 parallel jobs

# Display the best trial
print("Best trial:")
trial = study.best_trial
print(f"  Value (RMSE): {trial.value}")
print("  Params: ")
for key, value in trial.params.items():
    print(f"    {key}: {value}")

TypeError：XGBModel.fit() 获得意外的关键字参数“early_stopping_rounds”

我已经更新了所有内容，以确保我拥有所有更新的库。 我也通过各种法学硕士运行它，但没有帮助。 提前停止回合是正确的（我认为），但只是由于某种原因而爆炸。

谢谢！

Answer 1

EarlyStoppingRounds

已添加到

XGBoost

中的

1.4.0

。

如果版本低于 1.4.0，请使用以下命令升级 XGBoost：

pip install --upgrade xgboost

xgboost/releases/tag/v1.4.0