Fast and accurate content-based semantic search in 100M Internet Videos

Publication Type:
Conference Proceeding
MM 2015 - Proceedings of the 2015 ACM Multimedia Conference, 2015, pp. 49 - 58
Issue Date:
Filename Description Size
p49-jiang.pdfPublished version1.24 MB
Adobe PDF
Full metadata record
© 2015 ACM. Large-scale content-based semantic search in video is an interesting and fundamental problem in multimedia analysis and retrieval. Existing methods index a video by the raw concept detection score that is dense and inconsistent, and thus cannot scale to "big data" that are readily available on the Internet. This paper proposes a scalable solution. The key is a novel step called concept adjustment that represents a video by a few salient and consistent concepts that can be efficiently indexed by the modified inverted index. The proposed adjustment model relies on a concise optimization framework with interpretations. The proposed index leverages the text-based inverted index for video retrieval. Experimental results validate the efficacy and the efficiency of the proposed method. The results show that our method can scale up the semantic search while maintaining state-of-Theart search performance. Specifically, the proposed method (with reranking) achieves the best result on the challenging TRECVID Multimedia Event Detection (MED) zeroexample task. It only takes 0.2 second on a single CPU core to search a collection of 100 million Internet videos.
Please use this identifier to cite or link to this item: