|
| | | | |
| | | |
| Stat |
Members: 3645
Articles: 2'501'711
Articles rated: 2609
19 April 2024 |
|
| | | | |
|
Article overview
| |
|
|
ORD: Object Relationship Discovery for Visual Dialogue Generation | | Ziwei Wang
; Zi Huang
; Yadan Luo
; Huimin Lu
; | | Date: |
Mon, 15 Jun 2020 12:25:40 GMT (1825kb,D) | | Abstract: | With the rapid advancement of image captioning and visual question answering
at single-round level, the question of how to generate multi-round dialogue
about visual content has not yet been well explored.Existing visual dialogue
methods encode the image into a fixed feature vector directly, concatenated
with the question and history embeddings to predict the response.Some recent
methods tackle the co-reference resolution problem using co-attention mechanism
to cross-refer relevant elements from the image, history, and the target
question.However, it remains challenging to reason visual relationships, since
the fine-grained object-level information is omitted before co-attentive
reasoning. In this paper, we propose an object relationship discovery (ORD)
framework to preserve the object interactions for visual dialogue generation.
Specifically, a hierarchical graph convolutional network (HierGCN) is proposed
to retain the object nodes and neighbour relationships locally, and then
refines the object-object connections globally to obtain the final graph
embeddings. A graph attention is further incorporated to dynamically attend to
this graph-structured representation at the response reasoning stage. Extensive
experiments have proved that the proposed method can significantly improve the
quality of dialogue by utilising the contextual information of visual
relationships. The model achieves superior performance over the
state-of-the-art methods on the Visual Dialog dataset, increasing MRR from
0.6222 to 0.6447, and recall@1 from 48.48% to 51.22%. | | Source: | arXiv, 2006.8322 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
|
| |
|
|
|
|
News, job offers and information for researchers and scientists:
| |
REGISTER