Modality Coupling for Privacy Image Classification

Liu, Y; Huang, Y; Wang, S; Lu, W; Wu, H

Modality Coupling for Privacy Image Classification

Liu, Y Huang, Y Wang, S

Lu, W Wu, H

Permalink

Publisher:: IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Publication Type:: Journal Article
Citation:: IEEE Transactions on Information Forensics and Security, 2023, 18, pp. 4843-4853
Issue Date:: 2023-01-01

Closed Access

	Filename	Description	Size
	64. TIFS. 2023. Yonggang.pdf	Published version	16.05 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Liu, Y
dc.contributor.author	Huang, Y
dc.contributor.author	Wang, S https://orcid.org/0000-0003-1133-9379
dc.contributor.author	Lu, W
dc.contributor.author	Wu, H
dc.date.accessioned	2023-10-31T23:36:45Z
dc.date.available	2023-10-31T23:36:45Z
dc.date.issued	2023-01-01
dc.identifier.citation	IEEE Transactions on Information Forensics and Security, 2023, 18, pp. 4843-4853
dc.identifier.issn	1556-6013
dc.identifier.issn	1556-6021
dc.identifier.uri	http://hdl.handle.net/10453/172915
dc.description.abstract	Privacy image classification (PIC) has attracted increasing attention as it can help people make appropriate privacy decisions when sharing images. Most recently, some pioneer research efforts have been made to utilize multimodal information for PIC, since multi-modality can provide richer information than single modality. Those research efforts on multimodal PIC are under the assumption of independently identically distribution. However, connections between different modalities commonly exist in real-world cases. Taking the modalities of scene and object as example, in the scene of 'library/indoor', the object 'book jacket' resides with high probabilities. To this end, in this paper, a novel PIC approach, called CoupledPIC, is proposed to bridge this gap by comprehensively capturing the coupling relations between different modalities. In CoupledPIC, two submodules are designed to capture explicit and implicit coupling relations between different modalities respectively. The explicit modality coupling is learned with a tensor fusion networks based submodule, via the direct interaction of features. For the implicit modality coupling, a graph convolutional networks based submodule is proposed to learn on both the initial graphs and attention guided graphs, via information aggregation on graphs. Extensive experiments on the public benchmark, PicAlert, demonstrate the effectiveness of the proposed CoupledPIC, yielding significant improvement by modeling inter-modality coupling information.
dc.language	English
dc.publisher	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
dc.relation.ispartof	IEEE Transactions on Information Forensics and Security
dc.relation.isbasedon	10.1109/TIFS.2023.3301414
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	08 Information and Computing Sciences, 09 Engineering
dc.subject.classification	Strategic, Defence & Security Studies
dc.subject.classification	40 Engineering
dc.subject.classification	46 Information and computing sciences
dc.title	Modality Coupling for Privacy Image Classification
dc.type	Journal Article
utslib.citation.volume	18
utslib.for	08 Information and Computing Sciences
utslib.for	09 Engineering
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
utslib.copyright.status	closed_access	*
dc.date.updated	2023-10-31T23:36:43Z
pubs.publication-status	Published
pubs.volume	18

Abstract:

Privacy image classification (PIC) has attracted increasing attention as it can help people make appropriate privacy decisions when sharing images. Most recently, some pioneer research efforts have been made to utilize multimodal information for PIC, since multi-modality can provide richer information than single modality. Those research efforts on multimodal PIC are under the assumption of independently identically distribution. However, connections between different modalities commonly exist in real-world cases. Taking the modalities of scene and object as example, in the scene of 'library/indoor', the object 'book jacket' resides with high probabilities. To this end, in this paper, a novel PIC approach, called CoupledPIC, is proposed to bridge this gap by comprehensively capturing the coupling relations between different modalities. In CoupledPIC, two submodules are designed to capture explicit and implicit coupling relations between different modalities respectively. The explicit modality coupling is learned with a tensor fusion networks based submodule, via the direct interaction of features. For the implicit modality coupling, a graph convolutional networks based submodule is proposed to learn on both the initial graphs and attention guided graphs, via information aggregation on graphs. Extensive experiments on the public benchmark, PicAlert, demonstrate the effectiveness of the proposed CoupledPIC, yielding significant improvement by modeling inter-modality coupling information.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/172915