The most common cause of dementia globally is cerebrovascular disease (CVD). There is yet no perfect method for identifying patients with cardiovascular disease who have vascular-cognitive-impairment (VCI). Neuroimaging and clinical non-imaging data from 421 individuals with CVD are gathered in this study. We used this information to build a multimodal deep-learning framework that used methods like vision transformer and extreme-gradient-boosting (EDB). The framework’s final hybrid approach showed strong performance on both internal and external datasets using 2 neuroimaging characteristics and 6 clinical features. Additionally, our model was shown to have diagnostic performance that was analogous to that of expert doctors on a specific data set. Importantly, our model can pinpoint the specific areas of the brain and clinical characteristics that play a major role in the VCI diagnosis, making it easier to understand and use. This paper proposes a clinical decision support tool for identifying VCI in CVD patients that is both accurate and easy to understand. This research offers new understandings of how the kidneys age and indirectly supports clinical treatment decisions including dealing with kidney inflammation, stones, or tumors that may require nephrectomy, either partially or completely.