Cardiovascular disease, often treated as a cardiac illness, is a condition that occurs due to a change in the blood flow among the arteries in the heart. Although advanced techniques came into existence, we can observe that the mortality rate is increasing substantially among adults. Timely treatment and diagnosis are required to prevent heart failure. The lack of robustness in the accurate prediction of heart disease is a tough task due to insufficient data and the existence of outliers in the datasets. Several machine learning (ML) classifiers have been predominantly used in solving critical tasks and have proven their versatility by showing significant results. In this work, we have proposed a novel framework for early prediction of cardiovascular disease, where data pre-processing, outlier detection, predictive ML classifiers such as Naïve Bayes (NB), Support Vector Machine (SVM), k-Nearest Neighbours (k-NN), XGBoost (XGB)) were employed. In this work, the ensembling of various ML classifiers is also used as a method for improving heart disease prediction using k-fold cross validation (k-CV). The base classifiers are hypertuned using the grid search approach by considering numeric hyper parameters. The benchmark Cleveland heart disease dataset from the University of California (UCI) repository is consider for experiment. From the experiment it is found that, the proposed method outperforms the standard results in various evaluation metrics.