Face Recognition (FR) and surveillance video analytics are well-defined and solved problems in the applications of Computer Vision. FR aims to identify an already known person in a given image. Surveillance video analytics seeks to identify the occurrence of abnormal events or things in public places. But, recognizing the movements of most wanted criminals or suspects in public areas using FR systems with unclear surveillance video inputs is a very challenging problem. This work analyses the performance of three existing popular machine learning-based FR systems. They are (i) Viola–Jones detector, (ii) HOG-based FR, and (iii) PCA-based FR. This work analyzes the performance of these FR models on two different datasets. One is a benchmark dataset that has only the frontal view of the faces of various subjects. Another dataset we created with 10000 images. These images are collected from 50 subjects. From each subject, 200 images are taken from various angles. This work observes that the above models will improve their performance from 7 to 10% in terms of accuracy by training them on the proposed dataset.