Traffic rules violation is a serious crime that can also lead to accidents if not followed. There is a possible occurrence of human errors (corruption, unclear vision due to weather conditions) while monitoring the traffic, so Deep Learning models present a way to monitor traffic rules for two-wheelers if the driver doesn’t wear a helmet or triple rides. Within the context of this paper, we propose a model which can automatically detect and classify Indian vehicles from video recorded by a surveillance camera during frames extraction using the YOLOV3 (You Only Look Once) model which consists of Darknet53 as an architectural backbone that consists of 53 Convolutional Neural Network (CNN) layers. After performing object detection with vehicle classification, the model is supposed to segregate all the two-wheelers and perform helmet detection. For helmet detection, we proposed to build a custom-trained model from two variations of YOLO models—YOLOV3 proposed by AlexeyAB Darknet, and YOLOV8 proposed by Ultralytics using transfer learning. During the comparative analysis, we found that YOLOV3 provides a mean Average Precision (mAP) of 55.86% with an average precision of 82% for a confidence threshold of 0.25, and the latest updated version of the YOLOV8 model which uses CSPDarknet53 produced an mAP of 96.09% with an average precision of 96.90% for SGD optimizer and initial and final learning rate of 0.01 and batch size of 8. After detecting helmets from two-wheelers, automatic number plate recognition is performed using a similar YOLOV3 model followed by image super-resolution using ESRGAN (Enhanced Super-Resolution Generative Adversarial Networks) followed by Optical Character Recognition (OCR) using the tool Pytesseract. The deep learning models trained for performing object detection and segmentation are developed using transfer learning methodology to enhance the performance of pre-trained YOLO weights files to perform detections with lesser computational costs.