一次性将 pandas 数据帧随机分为几组，以进行 x 倍交叉验证

Question

假设我有一个包含 500 行的数据框。我想执行 10 倍交叉验证。因此，我需要将这些数据分为 10 组，每组包含 50 行。我也想将整个数据一次性分为 10 组，也是随机。

有没有办法使用 pandas、numpy 等库来做到这一点？

Answer 1

你可以使用sklearn的KFold：

import numpy as np
import pandas as pd
from sklearn.model_selection import KFold 

# Create dummy dataframe with 500 rows.
features = np.random.randint(1, 100, 500)
labels = np.random.randint(1, 100, 500)
df = pd.DataFrame(data = {"X": features, "Y": labels})

kf = KFold(n_splits=10, random_state=42, shuffle=True) # Define the split - into 10 folds.
kf.get_n_splits(df) # Returns the number of splitting iterations in the cross-validator.
print(kf) 

for train_index, test_index in kf.split(df):
    print("TRAIN:", train_index) 
    print("TEST:", test_index)
    X_train, X_test = df.loc[train_index, "X"], df.loc[test_index, "X"]
    y_train, y_test = df.loc[train_index, "Y"], df.loc[test_index, "Y"]

示例取自此处。