What Design Choices Actually Improve Deepfake Detector Performance?

Two research teams can use the exact same detection method and get very different results — not because one idea is better than the other, but because of implementation details like augmentation, image preprocessing, or training duration. A new study from the University of Bologna set out to isolate exactly which design choices actually move the needle, testing them systematically on the AI-GenBench benchmark.

Why Do Deepfake Detectors Perform So Inconsistently Across Studies?

• Reported success often depends less on the core detection idea than on implicit implementation choices
• Data augmentation, preprocessing, and training strategy are rarely tested in isolation
• This makes it hard to fairly compare methods or know which factor actually drove a result
• Many studies train on a single hand-picked generator and test on others, which doesn't reflect real deployment conditions

How Was This Study Designed?

• Built on AI-GenBench, a temporal benchmark that orders 36 image generators by release date, from 2017 GANs to 2024 diffusion models
• Tested across three pre-trained backbones: ResNet-50 CLIP, ViT-L/14 CLIP, and ViT-L/14 DINOv2
• Isolated five design dimensions: augmentation pipeline, augmentation multiplier and training duration, input processing at training and inference, multiclass training, and incremental update strategy
• Used average AUROC on the "Next Period" — performance on generators not yet seen — as the main success metric

Which Data Augmentation Strategy Works Best?

Three training pipelines were compared: a heavily perturbed baseline, a "mild" version, and one modeled on realistic social-media degradation with multiple compression passes.

• The realistic, evaluation-based pipeline (up to three JPEG compression passes with milder distortions) consistently beat the others, reaching 94.5% average AUROC versus 90.1% for the heavy baseline
• Excessively strong augmentation can be counterproductive
• Repeated JPEG compression during training appears especially important for preparing detectors for real-world image degradation

Does Longer Training or More Augmentation Diversity Matter More?

• Increasing the augmentation multiplier and increasing the number of training epochs produced roughly equivalent gains
• Larger models (ViT-L CLIP, DINOv2) plateau faster, often after just one or two epochs
• Smaller models like ResNet-50 CLIP kept improving with longer training schedules
• Training for four epochs while keeping augmentation diversity fixed offered the best performance-to-efficiency trade-off

Should Images Be Resized or Cropped for Training?

• Full-image resizing proved the most stable and reliable strategy across all three architectures
• Crop-based training only paid off for the largest backbone (DINOv2), and only when combined with a "mixed" evaluation that also uses the resized full image
• For smaller backbones, resizing alone remained the safest choice
• The optimal strategy is ultimately architecture-dependent

Does Training on Generator Labels (Multiclass) Help?

• Training purely as a multiclass problem (predicting which generator made the image) and converting to binary at the end performed worse than direct binary training
• Adding multiclass supervision as a secondary, lower-weighted objective (a dual-head setup with separate heads) matched or slightly beat plain binary training — with the bonus of also identifying which generator likely produced the image
• A distance-based scoring approach using class centroids consistently underperformed a standard classification head

How Should Detectors Be Updated as New Generators Emerge?

Retraining a detector from scratch every time a new generator appears is expensive. The study tested lighter alternatives.

• Naive fine-tuning on new data alone adapts quickly but causes severe forgetting of older generators
• Replay-based strategies, which retain a subset of past training examples, substantially reduce forgetting
• A harmonic replay schedule — gradually reducing the weight of older generators over time — offered the best trade-off between accuracy and computational cost
• Over a simulated 20-year deployment, replay strategies could cut computational cost by more than 80% compared to full retraining, with minimal accuracy loss

What's the Best Overall Configuration?

Combining the top-performing choices — the realistic augmentation pipeline, four training epochs, full-image resizing, and the DINOv2 backbone — produced a new state-of-the-art result of 97.36% average AUROC on the AI-GenBench Next Period metric.

Who Should Care About These Findings?

• Researchers designing new deepfake or AI-image detectors
• ML engineers building production detection pipelines that need to stay accurate over time
• Platforms and organizations that must periodically update detectors as new generators emerge, without unsustainable retraining costs

Frequently Asked Questions

What is the main finding of this study on deepfake detector design? Implementation details — such as augmentation strategy, training duration, and image preprocessing — often matter as much as the detection method itself, and a specific combination of these choices consistently improves generalization across different model architectures.

Which augmentation strategy works best for deepfake detection? A realistic pipeline that mimics social-media-style degradation, including multiple JPEG compression passes, consistently outperformed both heavier and milder augmentation strategies.

Is it better to resize or crop images when training a deepfake detector? Resizing the full image proved the most reliable approach across model architectures, though the largest models can gain a small additional benefit from also using image crops at inference time.

Does labeling images by generator (multiclass training) improve detection? Not on its own — direct binary training performed better. However, adding multiclass supervision as a secondary objective matched binary-only performance while also enabling attribution to a specific generator.

How can deepfake detectors be updated efficiently as new generators appear? Replay-based continual learning, especially a harmonic replay schedule that gradually reduces the weight of older generators, achieves performance close to full retraining at a fraction of the computational cost.

‍

Note: This article was prepared with the support of artificial intelligence 

‍

Discover more about the paper: click here

‍

Learn more about identifAI: talk to one of our experts

‍