Image-Relevant Entities Knowledge-Aware News Image Captioning - Ajankar S.; Dutta T. (My University, 2025)
News image captioning (NIC) generates entity-rich captions for news images via news article context. However, it inherits various challenges, like the presence of abstract semantic information based on named entities deteriorating the relationship between news images and the article. Due to the ambiguous relationship among image articles, the existing works struggle to exploit multimodal clues between text and images. To alleviate the aforementioned limitations, we proposed the image-relevant entities knowledge-aware NIC (IEK-NIC) novel framework. We propose to tweak the output of the model using the time-step-bounded entity constrained beam search algorithm for incorporating the entitiesâÂ€Â™ knowledge produced by the image-relevant entities generation method. The efficient usage of entities while generating captions plays a crucial role in enhancing performance. IEK-NIC shows an improvement in Consensus-based Image Description Evaluation score by a margin of 1.46 and 1.49 compared to the state of the art on the GoodNews and NYTimes800K datasets, respectively. © 1994-2012 IEEE.