In this paper, a novel feature extraction method based on deep and handcrafted features is proposed for radar remote sensing image retrieval. The main purpose is to capture the low-level and high-level image characteristics to enhance the representation of visual patterns. Initially, we applied a CNN to the RGB color image, which extracts high-level features and creates a feature descriptor known as FVCNN with a dimension of 128. We further divide the RGB image into its red (R), green (G), and blue (B) components to compute handcrafted features. Then, to find patterns within each component, we use sparse local ternary pattern (LTP) operators in vertical, horizontal, and diagonal directions. The LTP-based features are then combined to create an additional feature descriptor known as the ternary feature descriptor (FVT). The high dimension of FVT is reduced by Principal Component Analysis (PCA) to the top 128 features. The final feature descriptor (FVFinal), with a dimension of 256, is created by combining the feature descriptors FVCNN and FVT respectively. In order to capture a wider range of visual characteristics, this feature fusion aims to take advantage of the complementary strengths of both CNN and handcrafted-based features. To choose the most effective metric for the retrieval process, this paper evaluates seven similarity metrics including Bray–Curtis, Canberra, Chebyshev, City block, Correlation, Cosine, and Euclidean. The proposed method is validated by trials on the UCM dataset, which produced satisfactory retrieval outcomes.