Caption-aided Product Detection via Collaborative Pseudo-Label Harmonization
- Publisher:
- Institute of Electrical and Electronics Engineers (IEEE)
- Publication Type:
- Journal Article
- Citation:
- IEEE Transactions on Multimedia, 2022, PP, (99), pp. 1-13
- Issue Date:
- 2022-01-01
Closed Access
| Filename | Description | Size | |||
|---|---|---|---|---|---|
| Caption-Aided_Product_Detection_via_Collaborative_Pseudo-Label_Harmonization.pdf | Published version | 3.12 MB |
Copyright Clearance Process
- Recently Added
- In Progress
- Closed Access
This item is closed access and not available.
Product detection, which aims to localize products of interest in the advertising images, helps advance many potential E-commerce applications like product retrieval and recommendation. However, labeling a massive number of fine-grained product categories and accurate product boxes is costly and especially not practical since products are ever-changing on E-commerce websites. In this work, we step forward to train a fine-grained product detector solely supervised by the advertising captions, which are naturally available but often severely flawed and noisy. To reformulate the weakly supervised detection research into a real-world setting, we introduce a large-scale benchmark, named CapProduct , where more than 80,000 product image-caption pairs are collected from E-commerce websites. The fine-grained nature of products and noisy captions in CapProduct make it intractable to excavate valid category labels to train a weakly supervised object detector. To tackle this challenge, we propose a Co llaborative P seudo-L abel H armonization (CoPLH) framework that harmonizes self-mined pseudo labels via modeling the global co-occurrence relationships of products. We construct a collaborative co-occurrence graph based on all training samples to improve the reliability of caption-predicted pseudo-labels as well as benefit the self-training procedure in a weakly supervised setting. Extensive experiments on the CapProduct dataset demonstrate the effectiveness and the superiority of the proposed CoPLH over the state-of-the-art baselines.
Please use this identifier to cite or link to this item: