mRMR Feature Selection to Handle High Dimensional Datasets: Vertical Partitioning Based Iterative MapReduce Framework

Publications

mRMR Feature Selection to Handle High Dimensional Datasets: Vertical Partitioning Based Iterative MapReduce Framework

Author : Dr Vivek Yelleti

Year : 2024

Publisher : Springer Science and Business Media Deutschland GmbH

Source Title : Lecture Notes in Networks and Systems

Document Type :

Abstract

Feature selection stands out to be an important preprocessing step that is used to handle the uncertainty and vagueness in the data. In recent times, the minimum Redundancy and Maximum Relevance (mRMR) approach has been proven to be effective in obtaining the irredundant feature subset. Owing to the generation of voluminous datasets, it is essential to design scalable solutions using distributed/parallel paradigms. MapReduce solutions are proven to be one of the best approaches to designing fault-tolerant and scalable solutions. This work analyses the existing vertical partitioning MapReduce approaches for mRMR feature selection and identifies the limitations thereof. In the current study, we proposed VMR_mRMR, an efficient vertical partitioning-based approach using a memorization approach thereby overcoming the extant approaches limitations. The experiment analysis says that VMR_mRMR significantly outperformed extant approaches and achieved a better computational gain (C.G). We also conducted a comparative analysis with the horizontal partitioning approach HMR_mRMR to assess the strengths and limitations of the proposed approach.