A survey on deep neural network-based image captioning

作者：Liu, Xiaoxiao; Xu, Qingyang^*; Wang, Ning^*

来源：Visual Computer, 2019, 35(3): 445-470.

DOI：10.1007/s00371-018-1566-y

• Image captioning
• Image understanding
• Object detection
• Language model
• Attention mechanism
• Dense captioning

Image captioning is a hot topic of image understanding, and it is composed of two natural parts (look and language expression) which correspond to the two most important fields of artificial intelligence (machine vision and natural language processing). With the development of deep neural networks and better labeling database, the image captioning techniques have developed quickly. In this survey, the image captioning approaches and improvements based on deep neural network are introduced, including the characteristics of the specific techniques. The early image captioning approach based on deep neural network is the retrieval-based method. The retrieval method makes use of a searching technique to find an appropriate image description. The template-based method separates the image captioning process into object detection and sentence generation. Recently, end-to-end learning-based image captioning method has been verified effective at image captioning. The end-to-end learning techniques can generate more flexible and fluent sentence. In this survey, the image captioning methods are reviewed in detail. Furthermore, some remaining challenges are discussed.

• 出版日期2019-3
• 单位山东大学; 大连海事大学