Deepfake detection using Eulerian video magnification and Deep Learning

Resumo

The Internet witnesses millions of video views every minute in the contemporary landscape of extensive data proliferation and ubiquitous social media use. In this context, the burgeoning advancements in deepfake technologies introduce security and privacy concerns. These technologies facilitate manipulating video and audio content to an extent where someone can seamlessly replace one persons visage with anothers, or entirely synthetic videos can be crafted using real individuals voices and appearances, potentially deceiving viewers. Consequently, the malicious exploitation of deepfakes has caused apprehension due to its potential adverse societal repercussions, including, but not limited to, harm infliction, extortion, and reputational jeopardy. This study seeks, within the realm of deepfake detection, to evaluate the efficacy of Eulerian video magnification (EVM), a technique that accentuates subtle cues and motions, typically imperceptible to the naked eye. To this end, we propose the use of hybrid architectures comprising Convolutional Neural Networks (CNN) and Vision Transformers (ViT) with magnification techniques. Subsets of the FakeAVCeleb dataset, including authentic and manipulated videos, will train, validate, and test the model. The evaluation of the model will employ metrics such as accuracy, precision, recall, and the F1 score, with results compared to the existing literature.