Between the year 1994 and 2018, a considerable new and different metaheuristic optimization techniques have been presented in the literature for automated cryptanalysis of classical transposition cipher. This paper compares the performance of these new and different metaheuristic techniques. Three main comparison measures are considered to assess the performance of presented metaheuristics: effectiveness, efficiency and success rate. It is noteworthy that among the presented metaheuristics the performance of genetic algorithm technique is best with respect to all the measures.