The process of automatically finding and localizing the available anomalies (or defects) in the images of the products is known as Image Anomaly Detection and Localization (IADL). The IADL improves the efficiency of industrial quality inspection and ensures the desired quality level of the final products. Further, most of the supervised techniques are unsuitable for the IADL due to inherent data imbalance and ambiguity associated with the anomalies. Hence, this paper investigates key deep learning-based unsupervised IADL methods, such as Patch Distribution Modeling (PaDiM), Student-Teacher Feature Pyramid Matching (STFPM), Conditional Normalizing Flow (CFlow), Probabilistic Modeling of Deep Features for Out-of-Distribution and Adversarial Detection (DFM), and Deep Feature Kernel Density Estimation (DFKDE), for three publicly available bench-marked industrial defect detection datasets: MVTec AD, Visa and BTAD. Finally, a comparative analysis using both quantitative and qualitative performance metrics at the image as well as pixel levels is performed to draw some insightful conclusions.