|
| | | | |
| | | |
| Stat |
Members: 3662
Articles: 2'599'751
Articles rated: 2609
13 December 2024 |
|
| | | | |
|
Article overview
| |
|
|
Prioritizing Speech Test Cases | | Zhou Yang
; Jieke Shi
; Muhammad Hilmi Asyrofi
; Bowen Xu
; Xin Zhou
; DongGyun Han
; David Lo
; | | Date: |
1 Feb 2023 | | Abstract: | With the wide adoption of automated speech recognition (ASR) systems, it is
increasingly important to test and improve ASR systems. However, collecting and
executing speech test cases is usually expensive and time-consuming, motivating
us to strategically prioritize speech test cases. A key question is: how to
determine the ideal order of collecting and executing speech test cases to
uncover more errors as early as possible? Each speech test case consists of a
piece of audio and the corresponding reference text. In this work, we propose
PROPHET (PRiOritizing sPeecH tEsT), a tool that predicts potential
error-uncovering speech test cases only based on their reference texts. Thus,
PROPHET analyzes test cases and prioritizes them without running the ASR
system, which can analyze speech test cases at a large scale. We evaluate 6
different prioritization methods on 3 ASR systems and 12 datasets. Given the
same testing budget, we find that our approach uncovers 12.63% more wrongly
recognized words than the state-of-the-art method. We select test cases from
the prioritized list to fine-tune ASR systems and analyze how our approach can
improve the ASR system performance. Statistical tests show that our proposed
method can bring significantly larger performance improvement to ASR systems
than the existing baseline methods. Furthermore, we perform correlation
analysis and confirm that fine-tuning an ASR system using a dataset, on which
the model performs worse, tends to improve the performance more. | | Source: | arXiv, 2302.00330 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
|
| |
|
|
|
REGISTER