|
| | | | |
| | | |
| Stat |
Members: 3645
Articles: 2'506'133
Articles rated: 2609
27 April 2024 |
|
| | | | |
|
Article overview
| |
|
|
Attention-based Random Forest and Contamination Model | | Lev V. Utkin
; Andrei V. Konstantinov
; | | Date: |
8 Jan 2022 | | Abstract: | A new approach called ABRF (the attention-based random forest) and its
modifications for applying the attention mechanism to the random forest (RF)
for regression and classification are proposed. The main idea behind the
proposed ABRF models is to assign attention weights with trainable parameters
to decision trees in a specific way. The weights depend on the distance between
an instance, which falls into a corresponding leaf of a tree, and instances,
which fall in the same leaf. This idea stems from representation of the
Nadaraya-Watson kernel regression in the form of a RF. Three modifications of
the general approach are proposed. The first one is based on applying the
Huber’s contamination model and on computing the attention weights by solving
quadratic or linear optimization problems. The second and the third
modifications use the gradient-based algorithms for computing trainable
parameters. Numerical experiments with various regression and classification
datasets illustrate the proposed method. | | Source: | arXiv, 2201.02880 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
|
| |
|
|
|
|
News, job offers and information for researchers and scientists:
| |
REGISTER