|
| | | | |
| | | |
| Stat |
Members: 3645
Articles: 2'504'928
Articles rated: 2609
25 April 2024 |
|
| | | | |
|
Article overview
| |
|
|
ATST: Audio Representation Learning with Teacher-Student Transformer | | Xian Li
; Xiaofei Li
; | | Date: |
26 Apr 2022 | | Abstract: | Self-supervised learning (SSL) learns knowledge from a large amount of
unlabeled data, and then transfers the knowledge to a specific problem with a
limited number of labeled data. SSL has achieved promising results in various
domains. This work addresses the problem of segment-level general audio SSL,
and proposes a new transformer-based teacher-student SSL model, named ATST. A
transformer encoder is developed on a recently emerged teacher-student baseline
scheme, which largely improves the modeling capability of pre-training. In
addition, a new strategy for positive pair creation is designed to fully
leverage the capability of transformer. Extensive experiments have been
conducted, and the proposed model achieves the new state-of-the-art results on
almost all of the downstream tasks. | | Source: | arXiv, 2204.12076 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
|
| |
|
|
|
|
News, job offers and information for researchers and scientists:
| |
REGISTER