|
| | | | |
| | | |
| Stat |
Members: 3647
Articles: 2'515'668
Articles rated: 2609
13 May 2024 |
|
| | | | |
|
Article overview
| |
|
|
ESPnet-ONNX: Bridging a Gap Between Research and Production | | Masao Someki
; Yosuke Higuchi
; Tomoki Hayashi
; Shinji Watanabe
; | | Date: |
20 Sep 2022 | | Abstract: | In the field of deep learning, researchers often focus on inventing novel
neural network models and improving benchmarks. In contrast, application
developers are interested in making models suitable for actual products, which
involves optimizing a model for faster inference and adapting a model to
various platforms (e.g., C++ and Python). In this work, to fill the gap between
the two, we establish an effective procedure for optimizing a PyTorch-based
research-oriented model for deployment, taking ESPnet, a widely used toolkit
for speech processing, as an instance. We introduce different techniques to
ESPnet, including converting a model into an ONNX format, fusing nodes in a
graph, and quantizing parameters, which lead to approximately 1.3-2$ imes$
speedup in various tasks (i.e., ASR, TTS, speech translation, and spoken
language understanding) while keeping its performance without any additional
training. Our ESPnet-ONNX will be publicly available at
this https URL | | Source: | arXiv, 2209.09756 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
|
| |
|
|
|
REGISTER