|
| | | | |
| | | |
| Stat |
Members: 3645
Articles: 2'504'928
Articles rated: 2609
26 April 2024 |
|
| | | | |
|
Article overview
| |
|
|
Minimum Description Length Induction, Bayesianism, and Kolmogorov Complexity | | Paul Vitanyi
; Ming Li
; | | Date: |
27 Dec 1998 | | Journal: | IEEE Transactions on Information Theory, 46:2(2000), 446-464 | | Subject: | Learning; Artificial Intelligence; Computational Complexity; Information Theory; Logic in Computer Science; Probability; Data Analysis, Statistics and Probability ACM-class: E.4,F.2,H.3,I.2,I.5,I.7 | cs.LG cs.AI cs.CC cs.IT cs.LO math.PR physics.data-an | | Affiliation: | CWI and University of Amsterdam), Ming Li (University of Waterloo | | Abstract: | The relationship between the Bayesian approach and the minimum description length approach is established. We sharpen and clarify the general modeling principles MDL and MML, abstracted as the ideal MDL principle and defined from Bayes’s rule by means of Kolmogorov complexity. The basic condition under which the ideal principle should be applied is encapsulated as the Fundamental Inequality, which in broad terms states that the principle is valid when the data are random, relative to every contemplated hypothesis and also these hypotheses are random relative to the (universal) prior. Basically, the ideal principle states that the prior probability associated with the hypothesis should be given by the algorithmic universal probability, and the sum of the log universal probability of the model plus the log of the probability of the data given the model should be minimized. If we restrict the model class to the finite sets then application of the ideal principle turns into Kolmogorov’s minimal sufficient statistic. In general we show that data compression is almost always the best strategy, both in hypothesis identification and prediction. | | Source: | arXiv, cs.LG/9901014 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
|
| |
|
|
|
|
News, job offers and information for researchers and scientists:
| |
REGISTER