The high occurrence of crime in various cities worldwide has detrimental effects on both the victims and the communities they belong to. Although deep learning techniques are acknowledged for their effectiveness in predicting future events by analyzing past behaviors, those approaches have resulted in poor prediction accuracy for video surveillance data. To tackle this issue, a new framework called the Horned Lizard ZfNet Summarization Framework (HLZSF) was introduced in this research. The primary processes, like filtering, key frame extraction, crime events tracking, and crime prediction with classification, were performed. To filter the noise features continuously, the mathematical steps of the filtering process were processed in the hidden layer with the min-max scalar function. Moreover, the crime events tracking function is executed by processing the food-hunting behaviour of the horned lizard. In addition, the feature selection from the video frame is performed with the horned lizard skin colour changing as the best solution. The Python environment is adopted for this study to validate the video surveillance database. Here, the incorporation of the horned lizard’s best solution, which is skin changing behaviour based on a specific object in the deep network, has been employed to earn the finest feature selection and prediction outcome. The accuracy attained by the novel HLZSF is 97.87 %, and the recorded F-score is 97.88 %, precision 98.01 %, and recall 97.8 %, which is the finest outcome compared to alternative models.