Automatic recognition of violence and nonviolence activities in the crowd management system is a broad area of interest in today’s scenario. In this paper, we propose a hybrid combination of the Convolution Neural Networks (CNNs), and Long Short-Term Memory (LSTM) model to recognize violence/nonviolence activities in a crowded area. In the proposed approach a stream of video is applied to a pretrained Darknet-19 network, then a CNN with LSTM network is used to extract spatial and temporal features from the video. In the end, these spatial features are applied to a fully connected layer to identify the violence/nonviolence condition. The experimental results show that 98.1% accuracy was achieved in the case of video, and 97.8% accuracy was achieved in the case of the image frame by our proposed violence/nonviolence detection model.