Image to Modern Chinese Poetry Creation via a Constrained Topic-aware Model
- Publisher:
- ASSOC COMPUTING MACHINERY
- Publication Type:
- Journal Article
- Citation:
- ACM Transactions on Multimedia Computing, Communications and Applications, 2020, 16, (2)
- Issue Date:
- 2020-05-19
Closed Access
Filename | Description | Size | |||
---|---|---|---|---|---|
3381858.pdf | Published version | 3.68 MB |
Copyright Clearance Process
- Recently Added
- In Progress
- Closed Access
This item is closed access and not available.
© 2020 ACM. Artificial creativity has attracted increasing research attention in the field of multimedia and artificial intelligence. Despite the promising work on poetry/painting/music generation, creating modern Chinese poetry from images, which can significantly enrich the functionality of photo-sharing platforms, has rarely been explored. Moreover, existing generation models cannot tackle three challenges in this task: (1) Maintaining semantic consistency between images and poems; (2) preventing topic drift in the generation; (3) avoidance of certain words appearing frequently. These three points are even common challenges in other sequence generation tasks. In this article, we propose a Constrained Topic-aware Model (CTAM) to create modern Chinese poetries from images regarding the challenges above. Without image-poetry paired dataset, we construct a visual semantic vector to embed visual contents via image captions. For the topic-drift problem, we propose a topic-aware poetry generation model. Additionally, we design an Anti-frequency Decoding (AFD) scheme to constrain high-frequency characters in the generation. Experimental results show that our model achieves promising performance and is effective in poetry's readability and semantic consistency.
Please use this identifier to cite or link to this item: