Home Trans-ADENet: Transformer-based Attention-guided Deep Ensemble Network for high-dimensional data classification

Publications

Trans-ADENet: Transformer-based Attention-guided Deep Ensemble Network for high-dimensional data classification

Year : 2026

Publisher : Elsevier B.V.

Source Title : Array

Document Type :

Abstract

High-dimensional (HD) biomedical data, such as gene expression profiles and ECG signals, pose significant challenges for machine learning (ML) due to limited sample size, feature redundancy, and noisy distributions. Conventional models tend to overfit, whereas boosting and ensemble approaches struggle with irrelevant features. Deep autoencoders (DAE) reduce nonlinear dimensionality but miss complex dependencies, whereas transformers require large datasets to model long-range relationships through self-attention mechanisms. We propose a Transformer-based Attention-guided Deep Ensemble Network (Trans-ADENet) that integrates dimensionality reduction, attention-driven feature learning, and meta-level ensemble fusion in an end-to-end framework. A deep autoencoder compresses HD inputs into compact latent representations, refined by a Transformer Encoder with multi-head self-attention. Refined features are fed to diverse base classifiers (CatBoost, Support Vector Machine (SVM), TabNet, and Generalized Multi-Layer Perceptron (GMLP)), and their outputs are fused by a meta-MLP, which learns adaptive weights to yield robust predictions. Experiments on breast, leukemia, INCART2 and Thyroid-RNA datasets achieved 96.3%, 94.1%, 92.7% and 94.6% accuracy, surpassing state-of-the-art models in terms of accuracy, F1, precision, recall, and AUC. By combining representation learning, attention, and adaptive fusion, Trans-ADENet delivers accurate, interpretable classification for biomedical tasks.