Transformer models have been greatly improved recently, which has led to their extensive use in the process of captions for images, responding to questions regarding images, and the processing of both images and words. The current models use extensive computational power, time, and many configurable parameters for handling inquiries about remote working and detecting images. In this paper, we introduced an efficient and rapid visual question answering (VQA) system for satellite images using the Vision-and-Language Transformer framework (ViLT). Compared to the past models of remote sensing VQA, our approach improves the efficiency of the training process while maintaining robust image understanding. Our model utilizes a ViLT backbone consisting of immobilized transformer components. A customized block using dense layers with ReLU6 activation has been developed for categorization and the generation of answers from low-resolution images. This approach makes it easier to adapt to VQA tasks that involve several tunable parameters. Testing of the conventional remote sensing VQA (RSVQA) dataset shows that our system performs well with an overall accuracy of 86.26%. Moreover, this proposed model has lower operational expenses than the alternatives that require comprehensive retraining.