Apple might bring a rather dodgy feature into action tomorrow. A new tool will let Apple scan your photos stored in iCloud to check them against child sexual abuse material (CSAM). While Apple deserves an A+ for intent, the technology and process are quite malleable.
The claims have been made by cryptographer and security technologist Matthew D. Green, who works as an Associate Professor at the Johns Hopkins Information Security Institute. In a series of tweets, Green has shared Apple’s plans and what they may imply, if and when brought to action.
Green writes that multiple people have confirmed to him about Apple’s plan to launch a new scanning tool tomorrow. The “client-side tool” will run on the users’ devices and scan for CSAM. Any image marked by it will be reported back to Apple servers.
“The way this will work is that your phone will download a database of “fingerprints” for all of the bad images (child porn, terrorist recruitment videos etc.) It will check each image on your phone for matches.”
He mentions that the tool will use a specific perceptual hashtag to look for CSAM. Photos that match the hashtag will eventually be reported back to Apple’s servers.
These hashes have allegedly been created using “a new and proprietary neural hashing algorithm” that Apple has developed. Green also claims that Apple has received National Center for Missing & Exploited Children or NCMEC’s approval to use it.
Such a tool will understandably help catch and eventually bring down the distribution and production of child pornography, at least within the iOS ecosystem. What Greens is concerned about is the vast implication that such a tool brings.
The potential threats
Note that Apple will scan only the images stored in iCloud, meaning those which have already been shared with Apple. However, a tool that can scan your pictures does not sound like a very privacy-friendly thing to have.
Green points that out. In his tweets, he argues that such a client-side scanning mechanism can potentially be used to look for any sort of image on your device. The problem here is that the user will have no clue of the kind of hashtags or “fingerprints” being used during scanning. Anyone having access to the system can thus use any sort of fingerprint to gather a particular set of files from your library.
The whole question then turns to – how much do you trust Apple? If it would want to use the tool to look for something other than CSAM among your photos tomorrow, it can easily do so using the same tool.
Even if Apple uses it for the right purpose, in the right manner, these hashtags are susceptible to false matches. Green highlights that they are made to be “imprecise” so they can catch close matches, i.e. even if an image is cropped or altered in other forms.
Due to this, the system often mistakes a completely harmless image for abusive material. We have all seen that on Twitter and other social media platforms when they refuse to show a particular media because “it includes content you’ve chosen not to see.” Open it, and you see a puppy running around in a field of flowers.
Other than Apple misusing the tool, Green mentions his concern about threat actors. If someone manages to make collisions, or in other terms, fool the system to detect unrelated images, its effectiveness would be reduced to nothing.
The most unsettling thought, however, is that Apple might start using the tool in its encrypted services going forward. There is already a lot of pressure from law enforcement agencies around the globe to increase the monitoring of encrypted services. If Apple is directed to use the tool for such and related purposes within the encrypted network in the future, there would not be much left of Apple’s tall claims for privacy.