Deep learning has advanced medical imaging, but limited interpretability hinders clinical adoption. Class activation maps (CAM) provide visual explanations, yet methods such as Score-CAM are computationally expensive, requiring a forward pass for each activation map and limiting real-time applicability despite their high fidelity. To overcome this limitation, LowRank-CAM is proposed, which aggregates activation maps into a global matrix and applies singular value decomposition (SVD) to extract dominant spatial modes. The resulting top-r low-rank attention masks, with r ≪ K (r denotes the low-rank dimension and K is the total number of activation maps) replace per-channel perturbations and require only r forward passes through the classifier head. The resulting top-r low-rank attention masks, with r ≪ K, replace per-channel perturbations and require only r forward passes through the classifier head. This low-rank formulation substantially reduces complexity while preserving class-discriminatory importance. Experiments on Inceptionv3 musculoskeletal radiographs (MURA) demonstrate that LowRank-CAM achieves a 4.73 × speedup over ScoreCAM while maintaining comparable visual clarity and diagnostic relevance.