High blood pressure, or hypertension, has a direct association with retinal diseases, specifically hypertensive retinopathy (HR) and diabetic retinopathy (DR). HR is more prevalent in patients with severe and prolonged hypertension. Early identification of HR is crucial for assessing the risk of blindness. However, existing computerized techniques for diagnosing both HR and DR are limited. These methods often rely on conventional machine learning approaches, which are time-consuming and involve intricate image-processing steps. To address these challenges, we propose HDR-EfficientNet (HDR-EN), a deep learning algorithm to diagnose eye disorders effectively. This technique incorporates a spatial channel attention mechanism to enhance its ability to identify specific lesion regions and distinguish between various diseases. The HDR-EN model uses transfer learning methods to address the issues of imbalanced sample classes. Fractal dimension (FD) analysis is used to quantify differences in the retinal vessels, focusing on the optic cup and disc areas. FD values of the optic cup and disc play a crucial role in identifying HDR by revealing noticeable differences from healthy images, which are essential for precise classification. The results demonstrate high efficiency, with an average AUC of 0.98, an accuracy of 98%, a specificity of 96%, and a sensitivity of 95%. These findings indicate that the HDR-EN classifier could significantly diagnose HR and DR. In summary, the HDR-EN technique represents a deep learning-based method that offers improved accuracy and enhanced efficiency in detecting and categorizing retinal disorders.