Projects
Robust AI vs Real Image Detection
A frozen CLIP model and eight tiny probes that tell AI images from real ones, even after the internet has mangled them.
98.6%
Clean accuracy
Frozen threshold at 0.5, no tuning on eval data
0.996
AUROC
Above 0.996 under every transform
98.14%
Mean over 14 transforms
JPEG, blur, resize, noise, jitter, crop
15.2%
Of the 2B parameter budget
303,972,360 parameters, 6,152 of them trainable
Track 5: tell AI generated images from real ones, and keep doing it after JPEG, blur, resizing and noise. Scored on 500 COCO photos against 500 DALL-E 3 images, clean and under 14 degradations, with the threshold frozen in advance.
The machine
Mean accuracy under 14 real world degradations: 98.1%
What moved the number
How the detector improved
- Clean
- Mean over transforms
- Worst transform
Baseline Clean 0.811 Mean over transforms ≈0.75 Worst transform ≈0.73 Class weighted Clean 0.920 Mean over transforms 0.892 Worst transform 0.781 Augmented Clean 0.917 Mean over transforms 0.933 Worst transform 0.919 5 member ensemble Clean 0.978 Mean over transforms 0.974 Worst transform 0.968 8 member ensemble Clean 0.986 Mean over transforms 0.981 Worst transform 0.971
- Transform augmentation (my part). Three transformed copies of every AI training image. Mean accuracy under degradation went from 0.892 to 0.933.
- Class weighting. AUROC was already 0.983, but the probabilities sat too low for the frozen threshold. Upweighting the AI class moved them.
- Single generator probes (my part). One probe per generator disagree usefully. Mean transform accuracy rose from 97.39% to 98.14%.
What made it worse
A ninth probe (98.60% to 98.04%). Noise augmented features (1.7 points lost). A CLIP embedding drift feature, which failed exactly under noise.
Team 三T.