Detecting and removing polyps during colonoscopy in an early stage plays a crucial role in preventing colorectal cancer. Deep learning techniques have shown promise in automatically identifying polyps. However, the most current methods often prioritize accuracy over computational efficiency, which makes them hard to use in clinical settings with limited resources. This study introduces DYOLO, an improved deformable YOLOv7 architecture that effectively combines detection performance with computational efficiency for real-time polyp detection. The proposed method incorporates channel and spatial attention into the backbone network to improve discriminative feature learning, allowing the model to concentrate on essential polyp features while minimizing background noise. Deformable convolutions are integrated into the neck architecture to facilitate adaptive receptive field adjustment, thereby enhancing the model’s capacity to address various polyp morphologies observed in clinical settings. To evaluate the robustness of the DYOLO model, comprehensive testing was conducted using the Kvasir-SEG dataset, demonstrating that DYOLO achieves superior performance with 96.35% precision, 93.10% recall, and 95.8% mAP @ 0.5, outperforming existing methods.