Regular gradient descent can get trapped at a local bare minimum rather than a global bare minimum, causing a subpar network. In typical gradient descent, we just take all our rows and plug them into your exact same neural network, Examine the weights, and then modify them. Microsoft Material