| | |
| | |
Stat |
Members: 3667 Articles: 2'599'751 Articles rated: 2609
16 February 2025 |
|
| | | |
|
Article overview
| |
|
Adaptive Acceleration of Sparse Coding via Matrix Factorization | Joan Bruna
; Thomas Moreau
; | Date: |
1 Sep 2016 | Abstract: | Sparse coding remains a core building block in many data analysis and machine
learning pipelines. Typically it is solved by relying on generic optimization
techniques, that are optimal in the class of first-order methods for
non-smooth, convex functions, such as the Iterative Soft Thresholding Algorithm
and its accelerated version (ISTA, FISTA). However, these methods don’t exploit
the particular structure of the problem at hand nor the input data
distribution. An acceleration using neural networks was proposed in
citep{Gregor10}, coined LISTA, which showed empirically that one could achieve
high quality estimates with few iterations by modifying the parameters of the
proximal splitting appropriately.
In this paper we study the reasons for such acceleration. Our mathematical
analysis reveals that it is related to a specific matrix factorization of the
Gram matrix of the dictionary, in which unitary transformations leverage near
diagonalisation with small perturbations of the $ell_1$ norm. When this
factorization succeeds, we prove that the resulting splitting algorithm enjoys
an improved convergence bound with respect to the non-adaptive version.
Moreover, our analysis also shows that conditions for acceleration occur mostly
at the beginning of the iterative process, consistent with numerical
experiments. | Source: | arXiv, 1609.0285 | Services: | Forum | Review | PDF | Favorites |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
|
| |
|
|
|