There has been a sudden increase in demand for algorithms or models to correctly and accurately identify human emotions. The conformity for machines has come a long way from when smart machines capable of reaching a decision on their own were all that was expected of them, to machines capable of understanding what goes on in a person’s brain. Such autonomous agents can prove to be helpful not only in developing smarter machines but also in the field of medicine. Early prediction or recognizing brainwave patterns for epilepsy, seizures, manic depression, etc. is a key to achieve faster aid responses or prevention. In our work, we are limiting our focus to the most common practices used by researchers in this field, which is to obtain the electroencephalogram or EEG data, extract features and implement a classification algorithm. However, we are also trying to capitalize upon the massive improvements made in the field of supervised and unsupervised learning. The robust depth-wise separable convolution architecture called Xception has been implemented in this study to observe its performance as a feature extractor to the notoriously mutating EEG data. The EEG dataset being used in this study is open source. It is available in Kaggle and has three classes, namely positive, negative and neutral. We are implementing wavelet transform along with the Xception architecture to extract features from the dataset which are then classified using support vector machine. We achieve stellar results as a performance score of 98% can be observed for the measures accuracy, precision, recall as well as F1 score.