Abstract
Multimodal semantic image captioning is a cutting-edge approach that combines visual and textual modalities to generate accurate and contextually rich natural language descriptions of images. This method extends beyond traditional captioning by incorporating semantic understanding and capturing high-level concepts such as object attributes, relationships, and scene using temporal context. The proposed model effectively align visual content with multimodal linguistic meaning by combining visual features extracted from deep convolutional networks like ResNet-101 with semantic representations derived from textual embedding’s enriched by external knowledge bases and the language description generation is typically handled by sequence models like LSTM which is conditioned on the fused multimodal inputs. The semantic option helps the model to generate meaningful captions semantically aligned with the image content, leading to improved relevance, diversity, and human likeness. As Multimodal semantic image captioning progress with Flickr-30k data that consider multimodalities of information in the form of images and text descriptions trained with Adam Optimizer for 30 epochs. The proposed model shows greater performance with 63% Bleu and Meteor metric when compared with other models.