Abstract
Hyperspectral unmixing (HU) is a crucial challenge in applications such as remote sensing, involving the extraction of pure material spectral signatures (endmembers) from a hyperspectral image (HSI) and their related proportions (abundances) in each pixel. While nonlinear mixing models (NLMM) capture real-world complexities more effectively than linear mixing models (LMM), existing approaches often fail to adapt to spatially varying abundance distributions and hybrid spectral mixing, where both linear and nonlinear interactions influence the unmixing process. This paper introduces a novel Hybrid Hyperspectral Unmixing with Convolutional Autoencoder (CAE) and Temperature Scaling (H2U-CAETS). The proposed method features a parallel decoder architecture that integrates linear and nonlinear decoding paths to capture both types of spectral mixing. Additionally, it employs a hybrid loss function that optimally balances the mean squared error (MSE) and spectral angle distance (SAD) and a triangular thresholding method for temperature scaling, which enables more precise sparsity adjustments compared to traditional methods. Experimental evaluations on benchmark hyperspectral datasets (Samson, Urban, and DC) demonstrate that H2U-CAETS outperforms state-of-the-art techniques, achieving lower SAD and RMSE values. These results emphasise the robustness and adaptability of H2U-CAETS, paving the way for more effective hybrid hyperspectral unmixing in remote sensing applications.