|
| | | | |
| | | |
| Stat |
Members: 3645
Articles: 2'503'724
Articles rated: 2609
23 April 2024 |
|
| | | | |
|
Article overview
| |
|
|
Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery | | Yong-Hao Long
; Jie-Ying Wu
; Bo Lu
; Yue-Ming Jin
; Mathias Unberath
; Yun-Hui Liu
; Pheng-Ann Heng
; Qi Dou
; | | Date: |
3 Nov 2020 | | Abstract: | Automatic surgical gesture recognition is fundamentally important to enable
intelligent cognitive assistance in robotic surgery. With recent advancement in
robot-assisted minimally invasive surgery, rich information including surgical
videos and robotic kinematics can be recorded, which provide complementary
knowledge for understanding surgical gestures. However, existing methods either
solely adopt uni-modal data or directly concatenate multi-modal
representations, which can not sufficiently exploit the informative
correlations inherent in visual and kinematics data to boost gesture
recognition accuracies. In this regard, we propose a novel approach of
multimodal relational graph network (i.e., MRG-Net) to dynamically integrate
visual and kinematics information through interactive message propagation in
the latent feature space. In specific, we first extract embeddings from video
and kinematics sequences with temporal convolutional networks and LSTM units.
Next, we identify multi-relations in these multi-modal features and model them
through a hierarchical relational graph learning module. The effectiveness of
our method is demonstrated with state-of-the-art results on the public JIGSAWS
dataset, outperforming current uni-modal and multi-modal methods on both
suturing and knot typing tasks. Furthermore, we validated our method on
in-house visual-kinematics datasets collected with da Vinci Research Kit (dVRK)
platforms in two centers, with consistent promising performance achieved. | | Source: | arXiv, 2011.01619 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
|
| |
|
|
|
|
News, job offers and information for researchers and scientists:
| |
REGISTER