Gradient Descent
Optimization algorithm for training neural networks
Overview
An iterative optimization algorithm used to find the minimum of a function. In the context of training neural networks, gradient descent is used to adjust the model's internal parameters (weights and biases) to minimize the loss function, which is the difference between the model's predictions and the actual target values. The algorithm calculates the gradient of the loss function with respect to the parameters and iteratively updates the parameters in the direction of the negative gradient.
What is Gradient Descent?
Algorithm that optimizes neural network parameters.
How does it work?
Iteratively adjusts parameters to minimize errors.
Types?
Batch, mini-batch, and stochastic gradient descent.