With the notion of the Internet of Things (IoT), the concept of Internet of Medical Things (IoMT) emerges to improve the healthcare quality. The patients’ medical data collected and transmitted in IoMT is very sensitive in nature. Thus, there is a requirement of lightweight end-to-end mutual authentication protocol to ensure the secure communication between the patient and medical service provider. Recently, He et al proposed an anonymous authentication protocol for Wireless Body Area Networks. However, through security analysis we have identified some security weaknesses in their protocol. In this paper, we have designed an improved lightweight Elliptic Curve Cryptography based anonymous authentication protocol for IoMT, which has lightweight as compared to the He et al.’s protocol and also removes the security weaknesses of He et al.’s protocol. Further, the security evaluation (formal and informal) proves the security strength of the proposed protocol. Also, the comparative analysis with existing protocols demonstrates that the proposed authentication scheme is more robust and appropriate for IoMT applications.