Vehicle detection and number plate recognition approaches have been widely studied in recent years due to their wide applications. In this research paper, we propose a framework to ensure the entry of authorized vehicles in restricted areas such as the University campuses, townships, etc., where we are expecting the entry of a set of authorized vehicles. Certainly, unauthorized vehicles might be allowed to enter those areas after proper verification by the concerned people responsible for ensuring security. In the proposed approach, the admin should register all the authorized vehicles in a system with the essential attributes such as vehicle number, type, etc. A surveillance camera placed at the entrance will capture live videos. When there is a vehicle in the camera view, the image frames will be passed to an automatic number plate recognition module. The number plate recognition module will identify the same, and it will be matched with the details in the database to authorize the vehicle. This manuscript proposes a real-time and reliable approach for detecting and recognizing license plates based on morphology and template matching. To ensure the reliability of the system, a frame selection module will select the image frames with high quality, and even to improve the number plate recognition accuracy, the image will be enhanced using image enhancement techniques such as histogram equalization. The image enhancement techniques will help to provide better results even though the videos are taken with low lighting conditions. Further, we ensure that the vehicle type matches the number present in the database to prevent unauthorized access using fake number plates. The experimental study is conducted using videos taken under various environmental conditions such as lighting, slope, distance, and angle.