美文网首页
【ML】Parameters in Neural Network

【ML】Parameters in Neural Network

作者: 盐果儿 | 来源:发表于2022-07-01 02:49 被阅读0次

    1. Learning Rate:

    Definition

    Batch Size:

    Training Loss:

    Training Time:

    iteration:

    For instance, if the training set has 1000 samples, batch size = 10, then training the dataset need 100 iterations, training once means 1 epoch.

    Learning rate is \uparrow, step in every epoch \uparrow, training time \downarrow, problem: miss the lowest point.

    Learning rate is \downarrow, step in every epoch \downarrow, training time \uparrow, problem: training is too low, find the local lowest point.

    Learning Rate: https://en.wikipedia.org/wiki/Learning_rate

    Learning Rate是在哪个图上走的?LR for gradient descent, step, weight updates in order to minimize the network's loss function.

    学习率调整:

    https://blog.csdn.net/lty_sky/article/details/105223840

    局部最小值和鞍点:

    https://blog.csdn.net/m0_37957160/article/details/121913311

    相关文章

      网友评论

          本文标题:【ML】Parameters in Neural Network

          本文链接:https://www.haomeiwen.com/subject/yxegvrtx.html