On the Adam optimizer convergence

Investigation of the convergence of popular optimization algorithms like Adam, RMSProp and propose new variants of these methods which provably converge to optimal solution in convex settings.

Link: https://openreview.net/forum?id=ryQu7f-RZ
PDF: https://openreview.net/pdf?id=ryQu7f-RZ

#iclr2018 #neuralnetworks #optimizers