
On Lipschitz Regularization of Convolutional Layers using Toeplitz Matrix Theory
Araújo, Alexandre; Negrevergne, Benjamin; Chevaleyre, Yann; Atif, Jamal (2021), On Lipschitz Regularization of Convolutional Layers using Toeplitz Matrix Theory, 35th AAAI Conference on Artificial Intelligence, 2021-02, Vancouver, Canada
View/ Open
Type
Communication / ConférenceDate
2021Conference title
35th AAAI Conference on Artificial IntelligenceConference date
2021-02Conference city
VancouverConference country
CanadaMetadata
Show full item recordAuthor(s)
Araújo, AlexandreLaboratoire d'analyse et modélisation de systèmes pour l'aide à la décision [LAMSADE]
Negrevergne, Benjamin
Laboratoire d'analyse et modélisation de systèmes pour l'aide à la décision [LAMSADE]
Chevaleyre, Yann
Laboratoire d'analyse et modélisation de systèmes pour l'aide à la décision [LAMSADE]
Atif, Jamal
Laboratoire d'analyse et modélisation de systèmes pour l'aide à la décision [LAMSADE]
Abstract (EN)
This paper tackles the problem of Lipschitz regularization of Convolutional Neural Networks. Lipschitz regularity is now established as a key property of modern deep learning with implications in training stability, generalization, robustness against adversarial examples, etc. However, computing the exact value of the Lipschitz constant of a neural network is known to be NP-hard. Recent attempts from the literature introduce upper bounds to approximate this constant that are either efficient but loose or accurate but computationally expensive. In this work, by leveraging the theory of Toeplitz matrices, we introduce a new upper bound for convolutional layers that is both tight and easy to compute. Based on this result we devise an algorithm to train Lipschitz regularized Convolutional Neural Networks.Subjects / Keywords
Lipschitz; algorithmRelated items
Showing items related by title and author.
-
Araújo, Alexandre; Negrevergne, Benjamin; Chevaleyre, Yann; Atif, Jamal (2018) Communication / Conférence
-
Araújo, Alexandre; Negrevergne, Benjamin; Chevaleyre, Yann; Atif, Jamal (2019) Document de travail / Working paper
-
Vérine, Alexandre; Negrevergne, Benjamin; Rossi, Fabrice; Chevaleyre, Yann (2022) Communication / Conférence
-
Pinot, Rafaël; Meunier, Laurent; Yger, Florian; Gouy-Pailler, Cedric; Chevaleyre, Yann; Atif, Jamal (2022) Article accepté pour publication ou publié
-
Thanh Hai, Nguyen; Chevaleyre, Yann; Prifti, Edi; Sokolovska, Nataliya; Zucker, Jean-Daniel (2017) Communication / Conférence