Currently, adversarial attacks are typically designed for specific tasks, and although there are some task-agnostic attacks are generally less effective than task-specific ones. These attacks exploit the fact that CNN-based feature extractors cannot be reversed or inverted, making the downstream models vulnerable to these attacks. However, they are not optimally designed because they use the entire CNN to generate an adversarial example. This paper proposes a modified version of this approach called Faster Mimic and Fool (MaF), which requires less time and fewer resources to create an adversarial image. The experiment involved selecting 100 random FlickR 8K images and testing the attack on an Inception-V3-based captioning model. The results showed that Faster MaF achieved a Bleu-4 score that is 13.5% and 31.1% better than MaF and OIMO, respectively. Since Faster MaF requires knowledge of the CNN, it can be considered a grey-box attack.