Abstract
Most conventional double random phase encoding (DRPE)-based optical cryptosystems are vulnerable to various cryptographic attacks, including known-plaintext attacks (KPA) and chosen-plaintext attacks (CPA), thereby limiting their security. To address these limitations, this paper proposes a security-enhanced optical image encryption scheme in fractional Hermite transform (FrHMT) domain. The proposed approach combines DRPE architecture with fractional Hermite transform-domain modulation using two random phase masks applied respectively in the spatial and transform domains. The FrHMT introduces additional highly sensitive parameters, which significantly enlarge the key space and improve resistance against statistical and cryptographic attacks. The proposed scheme exhibits extremely high sensitivity to ciphertext modifications, where even minute changes in ciphertext pixels produce entirely different decrypted outputs. MATLAB simulations performed on grayscale, binary, and medical images demonstrate that the encrypted images exhibit noise-like distributions with negligible pixel correlations. The proposed scheme achieves high decryption accuracy with correlation coefficient values approaching unity, extremely low mean squared error, and high peak signal-to-noise ratio. Key sensitivity, histogram analysis, mesh plot analysis, and resistance against KPA and CPA further validate the robustness of the proposed cryptosystem. Due to its high sensitivity to ciphertext modifications, the scheme is particularly suitable for secure image transmission, tamper detection, authentication, and copyright protection applications.