紀要論文

Generalized DINO : Detection Transformers via Multimodal Models for Generalized Object Detection

eng

24321192

https://doi.org/10.15002/00030602

法政大学大学院情報科学研究科

法政大学大学院紀要. 情報科学研究科編

19

1

7

2024-03-24

Version of Record

Referring Expression Comprehension (REC) is a task in the realm of vision and language, aiming to identify objects in images based on provided descriptions. Classic REC methods, however, face challenges in handling expressions involving multiple targets or empty scenarios. In this paper, we study the limitations of existing REC methods, particularly in the context of Generalized Referring Expression Segmentation (GRES). In response, we propose Generalized DINO, a model that extends Transformer-based detectors by incorporating Region-Image Cross Attention (RIA) and Region-Language Cross Attention (RLA) mechanisms. This approach enables the detector to support arbitrary numbers of target object detection, overcoming the constraints of traditional REC methods. Comprehensive experiments on widely-used datasets such as RefCOCO/+/g and the GRES benchmark gRefCOCO showcase the superior performance of Generalized DINO in GRES tasks. The model outperforms even the robust RELA model, demonstrating a significant stride in handling expressions with multiple targets or empty scenarios. Our findings underscore the efficacy of Generalized DINO in enhancing the robustness and flexibility of REC models, contributing to multimodal information processing. The model's ability to handle complex language expressions involving multiple objects positions it as a valuable asset in applications like human-computer interaction and visual question answering.

Article

AA12746425