Abstract
In a first-of-its-kind study, we propose an online feature selection (OFS) framework for streaming data under big data paradigm, by proposing (i) parallel hybrid particle swarm optimization (PSO)-based wrapper, (ii) a robust method to handle high velocity and voluminous datasets, and (iii) two vigilance tests for detecting novelty. Our framework involves a continuous and adaptive learning process, by reducing the number of retrains of the wrapper. Moreover, it is scalable by virtue of the parallelization of the PSO and its variants/hybrids resulting in quick responses in real-time. We proposed BBPSO-L+TNS, a hybrid of bare-bones particle swarm optimization guided by logistic distribution (BPPSO-L) and threshold based neighbourhood search (TNS) heuristic, to achieve better exploitation capability and avoid entrapped in local optima. The findings demonstrate the robustness of the proposed streaming framework, yielding cost-effective solutions. Further, BBPSO-L+TNS outperformed the baseline algorithms.