我是这个DNN领域的新手,我对DNN中的Tunning超参数和其他参数感到厌倦,因为有很多参数需要调整,这就像多变量分析一样,不需要计算机的帮助。由于DNN内的变量众多,人类如何朝着使用DNN的任务可以达到的最高精度迈进。以及我们如何知道使用DNN可以达到什么精度,或者我必须放弃DNN?我搞不清楚了。感谢帮助。
我遇到的主要问题:
1. What are the limits of DNN / when we have to give up on DNN
2. What is the proper way of tunning without missing good parameter values
这是我在该领域学习理论所得的摘要。如果我是错误的或被误解了,请多多指教。您可以添加我错过的任何内容。根据我的知识,按重要性排序。
用于过度拟合-
1. reduce the number of layers
2. reduce the number of nodes of layers
3. add regularizers (l1/ l2/ l1-l2) - have to decide the factors
4. add dropout layers and -have to decide the dropout factor
5. reduce batch size
6. stop earlier
用于欠佳
1. increase the number of layers
2. increase number of nodes of layers
3. Add different types of layers (Conv, LSTM, ...)
4. add learning rate decay (decide the type and parameters for the type)
5. reduce the learning rate
除了我们通常可以做的以外,
1. number of epochs (by seeing what is happening while model training)
2. Adjust Learning Rate
3. batch normalization -for fast learning
4. initializing techniques (zero/ random/ Xavier / he)
5. different optimization algorithms
自动调整方法
- Gridsearchcv - but for this, we have to choose what we want to change and it takes a lot of time.
简短回答:您应该尝试很多