使用sklearn岭回归制作L曲线

Question

可视化ridge regression解的一种常见方法是L curve，它针对正则化参数的不同选择绘制平方误差的总和与脊峰惩罚的关系。 sklearn可以做到吗？

Answer 1

这里是一个纯粹的sklearn答案：

import numpy as np
from sklearn.linear_model import Ridge

alphas = np.logspace(-10, 10, 1000)
solution_norm = []
residual_norm = []

for alpha in alphas: 
    lm = Ridge(alpha=alpha)
    lm.fit(X, y)
    solution_norm += [(lm.coef_**2).sum()]
    residual_norm += [((lm.predict(X) - y)**2).sum()]

plt.loglog(residual_norm, solution_norm, 'k-')
plt.show()

其中X和y分别是您的预测变量和目标。

Answer 2

无法在scikit-learn中完成，但是Yellowbrick库提供了这种功能（随pip或conda一起安装）；将LassoCV示例从其conda改编为您的RidgeCV案例可得到：

documentation

import numpy as np
from sklearn.linear_model import RidgeCV
from yellowbrick.regressor import AlphaSelection
from yellowbrick.datasets import load_concrete

# Load the regression dataset
X, y = load_concrete()

# Create a list of alphas to cross-validate against
alphas = np.logspace(-10, 1, 40)

# Instantiate the linear model and visualizer
model = RidgeCV(alphas=alphas)
visualizer = AlphaSelection(model)
visualizer.fit(X, y)
visualizer.show()