The growing reliance on automobiles as the most common form of transportation and the increase in traffic has led to the need to implement essential safety measures. The driver’s alertness is critical to the safety of passengers in the vehicle. Existing methods for driver distraction detection face significant challenges as they rely on manually crafted features using traditional machine learning approaches, which are time-consuming to design, require domain expertise, and often fail to adapt to diverse real-world conditions. Although deep learning models have addressed some of these limitations by automatically extracting discriminative features, manually designed deep learning models struggle to handle complex driver distraction scenarios such as texting, yawning, and talking on the phone. The Deep Neural Network (DNN) models are well-known for their effectiveness across various computer vision tasks, including object localization and classification. However, manually designing efficient DNN models requires expertise. Even this may not always lead to an optimal model. To overcome this limitation, we propose the utilization of Neural Architecture Search (NAS) to design an automated method for generating DNN models. This work presents a framework for building a single-stage object detection model based on a NAS using an improved Genetic Algorithm (GA) for search strategy. The improved GA consists of Evaluation Correction based Selection (ECS) and Species Protection based Next Generation Population (SPNGP) to efficiently explore the search space and identify optimal backbone and training parameters. The designed search space includes the You Only Look Once (YOLO) backbone architecture parameters and associated training parameters. The experimental results suggest that the DNN model identified by the proposed approach has a smaller size and achieved better performance than the existing one-stage and two-stage object detection models, demonstrating the efficacy of our approach in designing driver distraction detection systems.