Topic 1 Question 18
An ML engineer has trained a neural network by using stochastic gradient descent (SGD). The neural network performs poorly on the test set. The values for training loss and validation loss remain high and show an oscillating pattern. The values decrease for a few epochs and then increase for a few epochs before repeating the same cycle. What should the ML engineer do to improve the training process?
Introduce early stopping.
Increase the size of the test set.
Increase the learning rate.
Decrease the learning rate.
ユーザの投票
コメント(5)
- 正解だと思う選択肢: D
The oscillating pattern of training and validation loss indicates that the learning rate is too high. A high learning rate causes the model to overshoot the optimal point in the loss landscape, leading to oscillations instead of convergence. Reducing the learning rate allows the model to make smaller, more precise updates to the weights, improving convergence.
A. Early stopping prevents overfitting by halting training when validation performance stops improving. However, it does not address the root cause of oscillating loss. B. The size of the test set does not affect the training dynamics or loss patterns. C. Increasing the learning rate would worsen the oscillations and prevent the model from converging.
👍 4motk1232024/12/09 - 正解だと思う選択肢: D
A. No, early stopping is for preventing overfitting B. No, increasing test will not help with oscillating loss C. No, increasing learning rate will make things worsening D. Oscillating loss in training is a sign that the training is not converging, this can happen when learning rate is too high. Reducing learning rate will help here
👍 4ninomfr642025/01/09 - 正解だと思う選択肢: D
oscillating = decrease the learning rate
👍 2GiorgioGss2024/11/28
シャッフルモード