We study the effect of using momentum (accumulation of earlier gradients) with the ensemble optimization method (EnOpt) for production optimization. Since its introduction to this field by Chen et al.

Deep learning models are usually trained with stochastic gradient descent-based algorithms, but these optimizers face inherent limitations, such as slow convergence and stringent assumptions for ...