Sugarcane is one of the most significant crops grown in India, which is the second highest producer after Brazil in the world. But one of the leaf diseases that affects the sugarcane yield is Red Rot. It must be found prior to effectively survive the illness and protect the crop. In this study, we will be training customized models using the CNN architecture to identify the red rot disease. By implementing MobileNet, Inception V3, VGG16, ResNet50 and a Hybrid Model for training them with the image dataset along with Adam’s, SGD (Stochastic Gradient Descent) and RMSProp (Root Mean Square Proportion) optimizers for enhancing the model’s performance. We evaluate each model and optimizer pairing extensively through testing and analysis. We built a Hybrid Model by the combination of Mobile Net and InceptionV3 with Adam Optimizer, attained an accuracy of 97.7%. By assisting in the creation of trustworthy techniques for early disease diagnosis in agriculture, our research ultimately aids farmers in safeguarding the yield and well-being of their crops.