Pneumonia is an illness that may have serious effects ranging from mild to deadly. It causes infection in the two air sacs of the lung. Chest X-ray picture interpretation for pneumonia identification is prone to subjectivity since it relies on the radiologist’s subjective judgment and experience, that may lead to uneven interpretations and diagnostic errors. This work presents deep learning architectures and interpretable techniques to give a detailed examination of pneumonia diagnosis. The main goals are to identify and classify pneumonia, evaluate different CNN (convolutional neural network) architectures, and enhance interpretability by using Lime segmentation. This will make deep learning models easier to comprehend by giving therapists concise, intelligible explanations for their predictions. An open-access dataset called Pneumonia was utilized for our study. The neural network models CNN, ResNet152V2, InceptionV3, MobileNetV2, DenseNet201, and VGG16 are all examined in this study. After thorough evaluation and hyperparameter adjustments, the models’ respective accuracy results were 87 %, 94 %, 89 %, 95%, 95%, and 97.95%. Our findings demonstrate that Lime segmentation enhances model interpretability, emphasizing the need for comprehensive methods for analyzing medical images to reliably and interpretably identify pneumonia.