我如何正确设置衰减学习率回调，并在xgboost中将其传递给自定义函数？

Question

我具有设置下降学习率的功能：

def learning_rate_005_decay_power_099(current_iter):
    base_learning_rate = 0.05
    lr = base_learning_rate  * np.power(.99, current_iter)
    return lr if lr > 1e-3 else 1e-3

现在我想将其传递给callback的train API中的xgboost参数。我正在尝试：

watchlist = [(xg_train, 'train'), (xg_test, 'test')]

num_round = params['num_round']
xgclassifier = xgb.train(params, xg_train, num_round, watchlist, early_stopping_rounds=early_stopping, callbacks = [xgb.callback.reset_learning_rate(learning_rate_005_decay_power_099)]);

但是我遇到此错误：

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-142-398cd3e1ab95> in <module>
     82 
     83             num_round = params['num_round']
---> 84             xgclassifier = xgb.train(params, xg_train, num_round, watchlist, early_stopping_rounds=early_stopping, callbacks = [xgb.callback.reset_learning_rate(learning_rate_005_decay_power_099)]);
     85             xgb_rounds.append(xgclassifier.best_iteration)
     86 

~/.local/lib/python3.6/site-packages/xgboost/training.py in train(params, dtrain, num_boost_round, evals, obj, feval, maximize, early_stopping_rounds, evals_result, verbose_eval, xgb_model, callbacks, learning_rates)
    214                            evals=evals,
    215                            obj=obj, feval=feval,
--> 216                            xgb_model=xgb_model, callbacks=callbacks)
    217 
    218 

~/.local/lib/python3.6/site-packages/xgboost/training.py in _train_internal(params, dtrain, num_boost_round, evals, obj, feval, xgb_model, callbacks)
     68                            end_iteration=num_boost_round,
     69                            rank=rank,
---> 70                            evaluation_result_list=None))
     71         # Distributed code: need to resume to this point.
     72         # Skip the first update if it is a recovery step.

~/.local/lib/python3.6/site-packages/xgboost/callback.py in callback(env)
    137         if context == 'train':
    138             bst, i, n = env.model, env.iteration, env.end_iteration
--> 139             bst.set_param('learning_rate', get_learning_rate(i, n, learning_rates))
    140         elif context == 'cv':
    141             i, n = env.iteration, env.end_iteration

~/.local/lib/python3.6/site-packages/xgboost/callback.py in get_learning_rate(i, n, learning_rates)
    128             new_learning_rate = learning_rates[i]
    129         else:
--> 130             new_learning_rate = learning_rates(i, n)
    131         return new_learning_rate
    132 

TypeError: learning_rate_005_decay_power_099() takes 1 positional argument but 2 were given

如果我使用完全相同的工作例程对lightgbm进行尝试，则效果很好：

xg_train = lgb.Dataset(X_train, label=y_train, silent = True)
            xg_test = lgb.Dataset(X_test, label=y_test, silent = True)

            watchlist = [xg_train, xg_test]
            names = ['train', 'test']

            xgclassifier = lgb.train(params, xg_train, num_round, valid_sets = watchlist, valid_names = names, early_stopping_rounds= early_stopping,
                                     callbacks=[lgb.reset_parameter(learning_rate=learning_rate_005_decay_power_099)]);

如何正确设置此操作？而且，什么使xgboost和lightgbm中的2个调用不同？

非常感谢您

Answer 1

f(boosting_round, num_boost_round)

您仅为参数boosting_round提供了值。

https://xgboost.readthedocs.io/en/latest/python/python_api.html#xgboost.callback.reset_learning_rate