Blogposts

• Why Stochastic Gradient Descent Stops Just Short of the Edge A closed-form sharpness gap explains a long-observed property of mini-batch training.
• Provable Acceleration of Nesterov's Momentum for Deep ReLU Networks A new objective class that makes Nesterov provably accelerated for non-trivial neural architectures.
• Provable Model-Parallel Distributed Principal Component Analysis with Parallel Deflation A self-correcting parallel deflation scheme for distributed PCA, with convergence guarantees.