Sensors installed at various assembly lines are becoming basic building blocks of industry 4.0 smart manufacturing environment. The data collected by sensors can be utilised to provide recommendation engine, automate the manufacturing process and also spot anomalies. Of these, automatic anomaly detection is a very significant task which detects misleading observations, data points, and/or events that deviate from the intended behaviour. The traditional approach to anomaly detection involves the use of more efficient and convoluted techniques to achieve higher accuracy. However, these techniques typically require much larger time and hence, is not suitable for real-time applications. In this work, an adaptive N-step anomaly detection technique is proposed wherein the number of steps (or modules) in the detection technique is based on the outlier percentage in the manufacturing process. A modular approach with techniques such as Density Based Spatial Clustering for Applications with Noise (DBSCAN), Isolation Forest (IF), Local Outlier Filter (LOF), etc., are used for automatic anomaly detection. Notably, the output of one step is fed as input to the next step, thereby making use of the knowledge gained in the previous step. The proposed adaptive approach results in >99% accuracy even when the outlier population is around 25%. Such high accuracy in real-time anomaly detection would serve as an important step towards having a Digital Twin model for smart manufacturing environment.