Computer vision works to interpret the content of images and videos. Shadows often cause problems because they can obscure important details and confuse object recognition and understanding of a scene. Existing shadow removal methods are generally complex, require a lot of resources, and struggle to adapt easily to real-time use or smaller devices. This paper introduces LightDiff-Shadow, a simple diffusion model for removing shadows from images in real-time. The method uses a two-part design that combines local denoising with understanding the overall scene. It uses a Reweighted Cross Attention and a Global Guided Sampling strategy to be more efficient and accurate. The model is trained from scratch using three standard datasets ISTD, ISTD+, and SRD. Experimental results show that our method achieves better performance in both qualitative and quantitative metrics outperforming recent state-of-the-art models such as ShadowFormer, Diff-Shadow, and Mask-ShadowGAN. The approach keeps textures sharp, illumination consistent, and overall visual quality good while using much less computing power. The findings suggest that simple setups can remove shadows well on devices with limited resources. This improves the accuracy with which a system interprets scenes and observes outdoor areas. It is highly beneficial for devices that need to process information instantly. The model produces output that demonstrate good visual quality, shows no artificial shadows, and preserves detail clarity under natural lighting conditions. This shows that it works in the real world.