Standard Gradient Descent is rarely enough for modern neural networks. In this guide, we trace the evolution of optimization algorithms—from the 'look-ahead' mechanism of Nesterov Accelerated Gradient to the adaptive learning rates of Adagrad and RMSProp. Finally, we demystify Adam to understand why it combines the best of both worlds.