Video anomaly detection (VAD) is a subject in computer vision that has a variety of applications such as civil protection, surveillance systems, intrusion detection, etc. Anomaly detection remains an ill-defined problem, despite its diverse applications. The proposed model for VAD is a Dual-Stream Variational Auto-Encoder (DSVAE), which consists of two stacked Variational Auto-Encoders (VAE) models. One model is a shallow generative model, i.e., Fully connected VAE (FCVAE) and the other is Skip connected VAE (SCVAE). The FCVAE model tries to learn the overall features of the model and rejects some of the unwanted features. The SCVAE attempts to extract the spatial and temporal features of the image frames in detail. SCVAE also uses the skip connection to connect the features from the encoder and decoder to minimize the information loss. The model is trained only on the normal video clips, and it tries to minimize the reconstruction error. Then the trained model is tested on the test videos. The reconstruction error is very high for testing video clips, resulting in a low regularity scores and a high anomaly. A properly set threshold regularity score segregates the anomaly frame from the normal one. Anomaly frames usually have low regularity score than the threshold value. The proposed DSVAE model is trained and tested on one of the widely used publicly available datasets, i.e., UCSD ped2, and the obtained performance results are found to be promising.