Accurate blood flow velocity measurement using Doppler ultrasound is critical for diagnosing cardiovascular diseases. The Doppler angle is a crucial step for accurately measuring blood velocity. The proposed study proposes a low-complexity deep learning framework integrated with Explainable AI (XAI) technique for Doppler angle estimation. For image segmentation, we evaluated several models, including VGG19, ResNet50, MobileNetV3, ResNet18, and EfficientNet-B0, assessing their Dice score accuracy and computational efficiency. The application of GradCAM XAI provides insights into model decision-making, crucial for enhancing diagnostic precision. Finally, a polynominial regression model facilitates Doppler angle estimation for assessments of blood flow dynamics. For best care scenario, we achieved a Dice score of 0.96 for segmented images and a mean error of -3.2 degrees for Doppler angle estimation. Overall, we propose a low-complexity model that enhances visual interpretation by segmenting arterial positions, uses for XAI for interpretability, automates Doppler angle estimation and is suitable for implementation on mobile devices.