A new report warns that the proliferation of child sexual abuse images on the internet could become much worse if something is not done to put controls on artificial intelligence tools that generate deepfake photos.
I was actually specifically avoiding all of those concerns in my reply. They’re valid, and others are discussing them on this thread, just not what my reply was about.
I was exclusively talking about how to identify if an image was generated by AI or was a real photo.
I was exclusively talking about how to identify if an image was generated by AI or was a real photo.
These images are being created with open source / free models. Whatever watermark feature the open source code has will simply be removed by the criminal.
Watermarking is like a lock on a door. Keeps honest people honest… which is useful, but it’s not going to stop any real criminals.
In this specific scenario, you wouldn’t want to remove the watermark.
The watermark would be the only thing that defines the content as “harmless” AI-generated content, which for the sake of discussion is being presented as legal. Remove the watermark, and as far as the law knows, you’re in possession of real CSAM and you’re on the way to prison.
The real concern would be adding the watermark to the real thing, to let it slip through the cracks. However, not only would this be computationally expensive if it was properly implemented, but I would assume the goal in marketing the real thing could only be to sell it to the worst of the worst, people who get off on the fact that children were abused to create it. And in that case, if AI is indistinguishable from the real thing, how do you sell criminal content if everyone thinks it’s fake?
Anyways, I agree with other commenters that this entire can of worms should be left tightly shut. We don’t need to encourage pedophilia in any way. “Regular” porn has experienced selection pressure to the point where taboo is now mainstream. We don’t need to create a new market for bored porn viewers looking for something shocking.
The real concern would be adding the watermark to the real thing, to let it slip through the cracks. However, not only would this be computationally expensive if it was properly implemented,
It wouldn’t be expensive, you could do it on a laptop in a few seconds.
Unless, of course, we decide only large corporations should be allowed to generate images and completely outlaw all of the open source / free image generation software - that’s not going to happen.
Most images are created with a “diffusion” model where you take an image, and run an algorithm that slightly modifies it. Over and over and over until you get what you want. You don’t have to (and commonly don’t - for the best results) start with a blank image. And you can run just a single pass, with the output being almost indistinguishable from the input.
This is a hard problem to solve and I think catching abuse after it happens is increasingly going to be more difficult. Better to focus on stopping the abuse from happening in the first place. E.g. by flagging and investigating questionable behaviour by kids in schools. That approach is proven and works well.
The image generation can be cheap, but I was imagining this sort of watermark wouldn’t be so much a visible part of the image, but an embedded signature that hashes the image.
Require enough PoW to generate the signature, and this would at least cut down the volumes of images created, and possibly limit them to groups or businesses with clusters that could be monitored, without clamping down on image generation in general.
A modified version of what you mentioned could work too, but where just these specific images have to be vetted and signed by a central authority using a private key. Image generation software wouldn’t be restricted for general purposes, but no signature on suspicious content and it’s off to jail.
I was actually specifically avoiding all of those concerns in my reply. They’re valid, and others are discussing them on this thread, just not what my reply was about.
I was exclusively talking about how to identify if an image was generated by AI or was a real photo.
These images are being created with open source / free models. Whatever watermark feature the open source code has will simply be removed by the criminal.
Watermarking is like a lock on a door. Keeps honest people honest… which is useful, but it’s not going to stop any real criminals.
In this specific scenario, you wouldn’t want to remove the watermark.
The watermark would be the only thing that defines the content as “harmless” AI-generated content, which for the sake of discussion is being presented as legal. Remove the watermark, and as far as the law knows, you’re in possession of real CSAM and you’re on the way to prison.
The real concern would be adding the watermark to the real thing, to let it slip through the cracks. However, not only would this be computationally expensive if it was properly implemented, but I would assume the goal in marketing the real thing could only be to sell it to the worst of the worst, people who get off on the fact that children were abused to create it. And in that case, if AI is indistinguishable from the real thing, how do you sell criminal content if everyone thinks it’s fake?
Anyways, I agree with other commenters that this entire can of worms should be left tightly shut. We don’t need to encourage pedophilia in any way. “Regular” porn has experienced selection pressure to the point where taboo is now mainstream. We don’t need to create a new market for bored porn viewers looking for something shocking.
It wouldn’t be expensive, you could do it on a laptop in a few seconds.
Unless, of course, we decide only large corporations should be allowed to generate images and completely outlaw all of the open source / free image generation software - that’s not going to happen.
Most images are created with a “diffusion” model where you take an image, and run an algorithm that slightly modifies it. Over and over and over until you get what you want. You don’t have to (and commonly don’t - for the best results) start with a blank image. And you can run just a single pass, with the output being almost indistinguishable from the input.
This is a hard problem to solve and I think catching abuse after it happens is increasingly going to be more difficult. Better to focus on stopping the abuse from happening in the first place. E.g. by flagging and investigating questionable behaviour by kids in schools. That approach is proven and works well.
The image generation can be cheap, but I was imagining this sort of watermark wouldn’t be so much a visible part of the image, but an embedded signature that hashes the image.
Require enough PoW to generate the signature, and this would at least cut down the volumes of images created, and possibly limit them to groups or businesses with clusters that could be monitored, without clamping down on image generation in general.
A modified version of what you mentioned could work too, but where just these specific images have to be vetted and signed by a central authority using a private key. Image generation software wouldn’t be restricted for general purposes, but no signature on suspicious content and it’s off to jail.