Ocean eddies have a significant effect on the maritime environment. They are necessary for carrying a variety of ocean traces across the ocean. Although deep learning algorithms for detecting eddies are a relatively new trend, it is still in their infancy. In this letter, a deep learning method for ocean eddy identification based on semantic segmentation is proposed. In semantic segmentation, understanding the context efficiently for pixel-level recognition is crucial. Two attention modules are proposed to tackle this problem. The proposed work consists of VGG16-based U-Net architecture with two attention modules to show a contextual correlation in the channel and spatial dimensions. Every pixel or channel adapts to include context from every other pixel or channel based on their correlations. Furthermore, a new residual path is proposed to replace the conventional skip connection between encoder and decoder modules. The findings of the experiments show that adopting an attention-based deep framework and new residual path improves the model performance over the existing state-of-the-art techniques.