In this paper, we propose a reinforcement learning (RL)-based attendance system (RLAS) for marking attendance of the students presented during a class using the frames captured by a video camera. The RLAS comprises of agent module and the environment module. In the agent module, we fine-tune the multi- task cascaded convolution network (MTCNN) module and the ArcFace module to identify the students present in class. The MTCNN module consists of two neural networks, the P-Network (P-Net) and the R-Network (R-Net). In the P-Net, we add 2 convolutional layers for extracting the latent features from the facial images of the students. Similarly, we modify the R-Net by adding two dense layers for detecting the bounding boxes from the frames captured by the video camera. Based on the latent features obtained from the fine-tuned MTCNN, the ArcFace identifies the students present in the class. The environment module of the RLAS uses the reward function to evaluate the output generated from the agent module. If the agent module correctly identifies all the students presented in the frames captured by the camera, then the reward function marks the attendance to those students. Else, the environment module back-propagates the error obtained from the reward function to the agent module. To evaluate the RLAS, we created a dataset of 1, 20, 000 different images of 2400 students studying at our university. In our experimental evaluation, we observed that the fine-tuned MTCNN along with the ArcFace provides the transfer learning mechanism to the RLAS. Therefore, the RLAS obtains less time complexity than the different variants of the MTCNN and CNN models.