Stochastic vs Batch Gradient Descent João Cláudio Nunes Carvalho One of the first concepts that a beginner comes across in the field of deep learning is gradient descent followed by various ways in which it can be implemented. Gradient descent is one of the most important concept used in the training of neural networks for supervised learning. Hence, it is important to understand it and the different ways in which it is to be carried out on the training sets. This post mostly deals with t h e different ways in which gradient descent is implemented on a training set. Thus, I will briefly go over the definition of the concept and then explain the advantages and disadvantages of all the possible ways. Gradient Descent This is an iterative optimization algorithm for finding the minimum of a function. The algorithm takes steps proportional to the negative gradient of the function at the current point [1]. In deep learning neural networks are trained by defining a loss function and opt...
Comentários