|
| | | | |
| | | |
| Stat |
Members: 3665
Articles: 2'599'751
Articles rated: 2609
23 January 2025 |
|
| | | | |
|
Article overview
| |
|
|
Rethinking the Video Sampling and Reasoning Strategies for Temporal Sentence Grounding | | Jiahao Zhu
; Daizong Liu
; Pan Zhou
; Xing Di
; Yu Cheng
; Song Yang
; Wenzheng Xu
; Zichuan Xu
; Yao Wan
; Lichao Sun
; Zeyu Xiong
; | | Date: |
2 Jan 2023 | | Abstract: | Temporal sentence grounding (TSG) aims to identify the temporal boundary of a
specific segment from an untrimmed video by a sentence query. All existing
works first utilize a sparse sampling strategy to extract a fixed number of
video frames and then conduct multi-modal interactions with query sentence for
reasoning. However, we argue that these methods have overlooked two
indispensable issues: 1) Boundary-bias: The annotated target segment generally
refers to two specific frames as corresponding start and end timestamps. The
video downsampling process may lose these two frames and take the adjacent
irrelevant frames as new boundaries. 2) Reasoning-bias: Such incorrect new
boundary frames also lead to the reasoning bias during frame-query interaction,
reducing the generalization ability of model. To alleviate above limitations,
in this paper, we propose a novel Siamese Sampling and Reasoning Network (SSRN)
for TSG, which introduces a siamese sampling mechanism to generate additional
contextual frames to enrich and refine the new boundaries. Specifically, a
reasoning strategy is developed to learn the inter-relationship among these
frames and generate soft labels on boundaries for more accurate frame-query
reasoning. Such mechanism is also able to supplement the absent consecutive
visual semantics to the sampled sparse frames for fine-grained activity
understanding. Extensive experiments demonstrate the effectiveness of SSRN on
three challenging datasets. | | Source: | arXiv, 2301.00514 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
|
| |
|
|
|
REGISTER