Improving SAM Requires R

Improving SAM Requires Re
Improving SAM Requires Rethinking its Optimization Formulation

This paper rethinks Sharpness-Aware Minimization (SAM), which is originally formulated as a zero-sum game where the weights of a network and a bounded perturbation try to minimize/maximize, respectively, the same differentiable loss. To fundamentally improve this design, we argue that SAM should ins…