Training a neural network is fundamentally an optimization problem. This blog explains the role of optimizers in deep learning, how gradient descent works, its batch, stochastic, and mini-batch variants, and why challenges like learning rate sensitivity, local minima, and saddle points motivate advanced optimization techniques.