| | |
| | |
Stat |
Members: 3645 Articles: 2'500'096 Articles rated: 2609
17 April 2024 |
|
| | | |
|
Article overview
| |
|
Sparsity vs. Statistical Independence in Adaptive Signal Representations: A Case Study of the Spike Process | Bertrand Benichou
; Naoki Saito
; | Date: |
7 Apr 2001 | Subject: | Numerical Analysis; Statistics MSC-class: 65T60, 65C60, 62M99, 94A17 | math.NA math.ST q-bio | Abstract: | Finding a basis/coordinate system that can efficiently represent an input data stream by viewing them as realizations of a stochastic process is of tremendous importance in many fields including data compression and computational neuroscience. Two popular measures of such efficiency of a basis are sparsity (measured by the expected $ell^p$ norm, $0 < p leq 1$) and statistical independence (measured by the mutual information). Gaining deeper understanding of their intricate relationship, however, remains elusive. Therefore, we chose to study a simple synthetic stochastic process called the spike process, which puts a unit impulse at a random location in an $n$-dimensional vector for each realization. For this process, we obtained the following results: 1) The standard basis is the best both in terms of sparsity and statistical independence if $n geq 5$ and the search of basis is restricted within all possible orthonormal bases in $R^n$; 2) If we extend our basis search in all possible invertible linear transformations in $R^n$, then the best basis in statistical independence differs from the one in sparsity; 3) In either of the above, the best basis in statistical independence is not unique, and there even exist those which make the inputs completely dense; 4) There is no linear invertible transformation that achieves the true statistical independence for $n > 2$. | Source: | arXiv, math.NA/0104083 | Services: | Forum | Review | PDF | Favorites |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser claudebot
|
| |
|
|
|
| News, job offers and information for researchers and scientists:
| |