Ensemble learning leverages the diversity among models to mitigate overfitting and underfitting issues. Insufficient diversity can lead to misclassification due to overfitting, while excessive diversity may yield random predictions from inaccurately performing models. Conventionally, ensembles train distinct models on the same dataset and aggregate their predictions. In our approach, we introduce a partitioning of the training dataset into k clusters, each containing related data. Through iterations, we randomly sample data from each cluster, combining them to create a new dataset called “ns,”which is used to train a model. After N iterations, an ensemble is constructed by combining the N trained models. Our proposed approach emphasizes the importance of diversity while addressing overfitting and underfitting concerns in ensemble learning. Experimental results validate the effectiveness of this methodology, highlighting its potential for improving ensemble performance.