Deepfake Detection Using Multi-Modal Fusion Combined with Attention Mechanism

Publications

Deepfake Detection Using Multi-Modal Fusion Combined with Attention Mechanism

Author : Dr Elakkiya E

Year : 2024

Publisher : Institute of Electrical and Electronics Engineers Inc.

Source Title : 4th International Conference on Sustainable Expert Systems, ICSES 2024 - Proceedings

Document Type :

Abstract

The proliferation of deepfake technology poses a significant challenge to the authenticity of digital content. This research explores the application of multimodal fusion techniques to enhance deepfake detection accuracy. By combining visual and audio features, the proposed method leverages the complementary nature of different data types to detect discrepancies introduced by deepfake manipulation. An attention mechanism is incorporated to focus on salient regions within each modality, further improving detection accuracy. Convolutional Neural Networks (CNNs) and Mel-Frequency Cepstral Coefficients (MFCCs) are employed for feature extraction, followed by feature fusion for deepfake detection. This approach demonstrates the effectiveness of multimodal fusion in combating the evolving threat of deepfake technology. By advancing deepfake detection techniques, this research contributes to safeguarding the integrity of digital content and preserving trust in media.