使用GridSearchCV进行Logistic回归时的精度计算警告

Question

我正在尝试使用 LogisticRegression 估计器运行 GridSearchCV 并记录模型准确性、精度、召回率、f1 指标。

但是，我在精度指标上遇到以下错误：

Precision is ill-defined and being set to 0.0 due to no predicted samples. 
Use `zero_division` parameter to control this behavior

我明白为什么会出现错误，因为 Kfold 分割中没有输出值等于 1 的预测。但是我不明白如何在 GridSearchCV （logistic_reg 变量）中将“zero_divison”具体设置为 1。

原代码

logistic_reg = GridSearchCV(estimator=LogisticRegression(penalty="l1", random_state=42, max_iter=10000), param_grid={
        "C": [1e-4, 5e-4, 1e-3, 5e-3, 1e-2, 5e-2, 1e-1, 5e-1, 1, 5, 10, 20],
        "solver": ["liblinear", "saga"]
        }, scoring=["accuracy", "precision", "recall", "f1"], cv=StratifiedKFold(n_splits=10), refit="accuracy")
    
logistic_reg_X_train = self.X_train.copy()
logistic_reg_X_train.drop(self.columns_removed, axis=1, inplace=True)
    
logistic_reg.fit(logistic_reg_X_train, self.y_train)
logistic_reg_results = pd.DataFrame(logistic_reg.cv_results_)

我尝试将“精度”更改为 precision_score(zero_division=1)，但这给了我另一个错误（

missing 2 required positional arguments: 'y_true' and 'y_pred'

）。我再次理解这一点，但在应用拟合方法之前没有定义 2 个缺失的参数。

如何为精度得分指标指定

1zero_division

参数？

编辑

我不明白的是，我在train_test_split方法中对y数据进行了分层，并在GridSearchCV中使用了StratifedKFold。我对此的理解是，训练/测试数据将具有相同的 y 值分割比例，并且在交叉验证期间也应该发生相同的情况。这意味着在gridsearchcv样本中，数据的y值应同时为0和1，因此精度不能等于0（模型将能够计算TP和FP，因为样本测试数据包含y等于1的样本）。我不知道从这里该去哪里。

Answer 1

通过进一步阅读此问题，我的理解是发生错误是因为并非 y_test 中的所有标签都出现在 y_pred 中。我的数据并非如此。

我使用 G.Anderson 的评论删除了警告（但它没有回答我的问题）

• 创建了新的 custom_scorer 对象

• 创建了 customer_scoring 字典

• 更新了 GridSearchCV 评分和改装参数

  from sklearn.metrics import precision_score, make_scorer
  
  precision_scorer = make_scorer(precision_score, zero_division=0)
  
  custom_scoring = {"accuracy": "accuracy", "precision": precision_scorer, "recall": "recall", "f1": "f1"}
  
  logistic_reg = GridSearchCV(estimator=LogisticRegression(penalty="l1", random_state=42, max_iter=10000), param_grid={
        "C": [1e-4, 5e-4, 1e-3, 5e-3, 1e-2, 5e-2, 1e-1, 5e-1, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20],
        "solver": ["liblinear", "saga"]
        }, scoring=custom_scoring, cv=StratifiedKFold(n_splits=10), refit="accuracy")
  

编辑 - 回答上述问题

我使用 GridSearchCV 来找到模型的最佳超参数。为了查看每个分割的模型指标，我创建了一个具有最佳超参数的 StratifedKFold 估计器，然后自行进行交叉验证。这没有给我任何精确的警告信息。我不知道为什么 GridSearchCV 给我一个警告，但至少这种方式有效！！！

注意：我从下面的方法和上面问题中的GridSearchCV得到相同的结果。

skf = StratifiedKFold(n_splits=10)
logistic_reg_class_skf = LogisticRegression(penalty="l1", max_iter=10000, random_state=42, C=5, solver="liblinear")
    
logistic_reg_class_score = []
                    
for train, test in skf.split(logistic_reg_class_X_train, self.y_train):
        
    logistic_reg_class_skf_X_train = logistic_reg_class_X_train.iloc[train]
    logistic_reg_class_skf_X_test = logistic_reg_class_X_train.iloc[test]
    logistic_reg_class_skf_y_train = self.y_train.iloc[train]
    logistic_reg_class_skf_y_test = self.y_train.iloc[test]
        
    logistic_reg_class_skf.fit(logistic_reg_class_skf_X_train, logistic_reg_class_skf_y_train)
    logistic_reg_skf_y_pred = logistic_reg_class_skf.predict(logistic_reg_class_skf_X_test)
        
    skf_accuracy_score = metrics.accuracy_score(logistic_reg_class_skf_y_test, logistic_reg_skf_y_pred)
    skf_precision_score = metrics.precision_score(logistic_reg_class_skf_y_test, logistic_reg_skf_y_pred)
    skf_recall_score = metrics.recall_score(logistic_reg_class_skf_y_test, logistic_reg_skf_y_pred)
    skf_f1_score = metrics.f1_score(logistic_reg_class_skf_y_test, logistic_reg_skf_y_pred)

    logistic_reg_class_score.append([skf_accuracy_score, skf_precision_score, skf_recall_score, skf_f1_score])

    classification_results = pd.DataFrame({"Algorithm": ["Logistic Reg Train"], "Accuracy": [0.0], "Precision": [0.0],
                                            "Recall": [0.0], "F1 Score": [0.0]})
    
    for i in range (0, 10):
        classification_results.loc[i] = ["Logistic Reg Train", logistic_reg_class_score[i][0], logistic_reg_class_score[i][1],
                                         logistic_reg_class_score[2][0], logistic_reg_class_score[3][0]]

Answer 2

就我而言，我收到此错误是因为某些超参数导致模型的正面预测为 0。对于这些模型，精度是不确定的，因为既没有真阳性也没有假阳性。

设置

error_score=0.0

会默默地将这些模型的精度设置为 0。