An approach to identify captioning keywords in an image using LIME

Abstract

Machine Learning models are being increasingly deployed to tackle real-world problems in various domains like healthcare, crime and education among many others. However, most of the models are practically "black-boxes": although they may provide accurate results, they are unable to provide any conclusive reasoning for those results. In order for these decisions to be trusted, they must be explainable. Explainable AI, or XAI refers to methods and techniques in the application of AI such that the results of the solution are understandable by human experts. This paper focuses on the task of Image Captioning, and tries to employ XAI techniques such as LIME (Local Interpretable Model-Agnostic Explanations) to explain the predictions of complex image captioning models. It visually depicts the part of the image corresponding to a particular word in the caption, thereby justifying why the model predicted that word. © 2021 IEEE.