Exploiting textual queries for dynamically visual disambiguation

Elsevier BV
Publication Type:
Journal Article
Pattern Recognition, 2020, pp. 107620-107620
Issue Date:
Filename Description Size
1-s2.0-S0031320320304234-main.pdfPublished version2.92 MB
Adobe PDF
Full metadata record
© 2020 Due to the high cost of manual annotation, learning directly from the web has attracted broad attention. One issue that limits the performance of current webly supervised models is the problem of visual polysemy. In this work, we present a novel framework that resolves visual polysemy by dynamically matching candidate text queries with retrieved images. Specifically, our proposed framework includes three major steps: we first discover and then dynamically select the text queries according to the keyword-based image search results, we employ the proposed saliency-guided deep multi-instance learning (MIL) network to remove outliers and learn classification models for visual disambiguation. Compared to existing methods, our proposed approach can figure out the right visual senses, adapt to dynamic changes in the search results, remove outliers, and jointly learn the classification models. Extensive experiments and ablation studies on CMU-Poly-30 and MIT-ISD datasets demonstrate the effectiveness of our proposed approach.
Please use this identifier to cite or link to this item: