|
| | | | |
| | | |
| Stat |
Members: 3645
Articles: 2'506'133
Articles rated: 2609
26 April 2024 |
|
| | | | |
|
Article overview
| |
|
|
High-performance Kernel Machines with Implicit Distributed Optimization and Randomization | | Vikas Sindhwani
; Haim Avron
; | | Date: |
3 Sep 2014 | | Abstract: | Complex machine learning tasks arising in several domains increasingly
require "big models" to be trained on "big data". Such models tend to grow with
the complexity and size of the training data, and do not make strong parametric
assumptions upfront on the nature of the underlying statistical dependencies.
Kernel methods constitute a very popular, versatile and principled statistical
methodology for solving a wide range of non-parametric modelling problems.
However, their storage requirements and high computational complexity poses a
significant barrier to their widespread adoption in big data applications. We
propose an algorithmic framework for massive-scale training of kernel-based
machine learning models. Our framework combines two key technical ingredients:
(i) distributed general purpose convex optimization for a class of problems
involving very large but implicit datasets, and (ii) the use of randomization
to significantly accelerate the training process as well as prediction speed
for kernel-based models. Our approach is based on a block-splitting variant of
the Alternating Directions Method of Multipliers (ADMM) which is carefully
reconfigured to handle very large random feature matrices only implicitly,
while exploiting hybrid parallelism in compute environments composed of loosely
or tightly coupled clusters of multicore machines. Our implementation supports
a variety of machine learning tasks by enabling several loss functions,
regularization schemes, kernels, and layers of randomized approximations for
both dense and sparse datasets, in a highly extensible framework. We study the
scalability of our framework on both commodity clusters as well as on
BlueGene/Q, and provide a comparison against existing sequential and parallel
libraries for such problems. | | Source: | arXiv, 1409.0940 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
|
| |
|
|
|
|
News, job offers and information for researchers and scientists:
| |
REGISTER