This. Matter of time until the (innocent) non-nude photos of people's children suddenly get blurred and red-flagged. It could even be a clothed child wearing a salmon-colored (skin-colored) clothes, which could trick the CSAM algorithm into thinking the child was unclothed.
And then Child Protection Services and local police suddenly comes knocking at the Parents' door.... hmm.... could be a tragic scenario.
Think more like checksumming and less like facial recognition. A checksum is a rote computation on the contents of your file, if anything is different, the checksum changes. Facial recognition is making a subjective judgement about "what looks like what".
The
hashing here isn't asking "does this look like a child being abused?", the hashing is asking "is this image on a list of specific images of abuse known to be in circulation?". It is not extrapolating and looking for new images, only images that are already known to be circulating.
Unlike a basic checksum, the image is manipulated before hashing to normalize for things like resizing and color table changes, to prevent simple image manipulations for fooling the hash. These are image manipulations, not machine learning inferences, so are not "recognition" tasks.
Like a checksum, information is lost in the hashing-- a complex image is reduced to a long number. The upside is that means the image can't be recreated from the hash. It also means that there is a remote chance that two images reduce to the same long number-- but that doesn't mean they'll look anything alike to the human eye, or that they contain "similar" content.
Because of the risk of false positives, there's a human check involved before forwarding to law enforcement. The question they are answering is "is this objectively the same image as the CSAM image that produced the hash". They are not answering "is this subjectively an image of child abuse", but it doesn't much matter because false positives are far more likely to be pictures of architecture or food than child abuse.
That also means that there's no reason to think that if you trigger a false positive that your image is "close to being bad"-- it would just be any other random image.
