Apple is raising privacy concerns with its devices.
Andrew Hoyle/CNET

Apple has long presented itself as a bastion of security, and as one of the only tech companies that truly cares about user privacy. But a new technology designed to help an iPhone, iPad or Mac computer detect child exploitation images and videos stored on those devices has ignited a fierce debate about the truth behind Apple’s promises.

On Aug. 5, Apple announced a new feature being built into the upcoming iOS 15, iPad OS 15, WatchOS 8 and MacOS Monterey software updates, designed to detect if people have child exploitation images or videos stored on their device. It’ll do this by converting images into unique bits of code, known as hashes, based on what they depict. The hashes are then checked against a database of known child exploitation content that’s managed by the National Center for Missing and Exploited Children. If a certain number of matches are found, Apple is then alerted and may further investigate.

Apple said it developed this system to protect people’s privacy, performing scans on the phone and only raising alarms if a certain number of matches are found. But privacy experts, who agree that fighting child exploitation is a good thing, worry that Apple’s moves open the door to wider uses that could, for example, put political dissidents and other innocent people in harm’s way.

“Even if you believe Apple won’t allow these tools to be misused there’s still a lot to be concerned about,” tweeted Matthew Green, a professor at Johns Hopkins University, who’s worked on cryptographic technologies.

Apple’s new feature, and the concern that’s sprung up around it, represent an important debate about the company’s commitment to privacy. Apple has long promised that its devices and software are designed to protect users’ privacy. The company even dramatized that with an ad it hung just outside the convention hall of the 2019 Consumer Electronics Show, which said, “What happens on your iPhone stays on your iPhone.”

“We at Apple believe privacy is a fundamental human right,” Apple CEO Tim Cook has often said.

Apple’s scanning technology is part of a trio of new features the company is planning for this fall. Apple also is enabling its Siri voice assistant to offer links and resources to people it believes may be in a serious situation, such as a child in danger. Advocates had been asking for that type of feature for a while.

It’s also adding a feature to its messages app to proactively protect children from explicit content, whether it’s in a green-bubble SMS conversation or blue-bubble iMessage encrypted chat. This new capability is specifically designed for devices registered under a child’s iCloud account and will warn if it detects an explicit image being sent or received. Like with Siri, the app will also offer links and resources if needed.

Apple’s system will also alert children about explicit images being sent or received on its messages app.
Apple

There’s a lot of nuance involved here, which is part of why Apple took the unusual step of releasing research papers, frequently asked questions and other information ahead of the planned launch.

Here’s everything you should know:

Why is Apple doing this now?

Apple’s iCloud photo library syncing feature synchronizes images and videos between a person’s devices and the company’s servers.
Apple

The tech giant said it’s been trying for a while to find a way to help stop child exploitation. The National Center for Missing and Exploited Children received more than 65 million reports of material last year. Apple said that’s way up from the 401 reports 20 years ago.

“We also know that the 65 million files that were reported is only a small fraction of what is in circulation,” said Julie Cordua, head of Thorn, a nonprofit fighting child exploitation that supports Apple’s efforts. She added that US law requires tech companies to report exploitative material if they find it, but it doesn’t compel them to search for it.

Other companies do actively search for such photos and videos. Facebook, Microsoft, Twitter and Google (and its YouTube subsidiary) all use various technologies to scan their systems for any potentially illegal uploads.

What makes Apple’s system unique is that it’s designed to scan our devices, rather than the information stored on the company’s servers.

The hash scanning system will be applied only to photos stored in iCloud Photo Library, which is a photo syncing system built into Apple devices. It won’t hash images and videos stored in the photos app of a phone, tablet or computer that isn’t using iCloud Photo Library. So, in a way, people can opt out if they choose not to use Apple’s iCloud photo syncing services.

Read more: Apple software head says plan to scan iPhones for child abuse images is ‘misunderstood’

Could this system be abused?

China aggressively censors political speech and imagery.
Getty Images

The question isn’t whether Apple should do what it can to fight child exploitation. It’s whether the company should use this method.

The slippery slope concern privacy experts have raised is whether Apple’s tools could be twisted into surveillance technology against dissidents. Imagine if the Chinese government were able to somehow secretly add data corresponding to the famously suppressed Tank Man photo from the 1989 pro-democracy protests in Tiananmen Square to Apple’s child exploitation content system.

Apple said it designed features to keep that from happening. The system doesn’t scan photos, for example — it checks for matches between hash codes. The hash database is also stored on the phone, not a database sitting on the internet. Apple also noted that because the scans happen on the device, security researchers can more easily audit the way it works.

Is Apple rummaging through my photos?

We’ve all seen some version of it: The baby in the bathtub photo. My parents had some of me, I have some of my kids, and it was even a running gag on the 2017 Dreamworks animated comedy The Boss Baby.

Apple says those images shouldn’t trip up its system. Because Apple’s system converts our photos to these hash codes, and then checks them against a known database of child exploitation videos and photos, the company isn’t actually scanning our stuff. The company said the likelihood of a false positive is less than one in 1 trillion per year.

“In addition, any time an account is flagged by the system, Apple conducts human review before making a report to the National Center for Missing and Exploited Children,” Apple wrote on its site. “As a result, system errors or attacks will not result in innocent people being reported to NCMEC.”

Is Apple reading my texts?

Apple isn’t applying its hashing technology to our text messages. That, effectively, is a separate system. Instead, with text messages, the setup applies to users marked as children in their iCloud accounts, and Apple is alerting them when they’re about to send or receive an explicit image. The children can still view the image, and if they do, parents will be alerted.

“The feature is designed so that Apple does not get access to the messages,” Apple said.

What does Apple say?

Apple maintains that its system is built with privacy in mind, with safeguards to keep the company from knowing the contents of our photo libraries and to minimize the risk of misuse.

In an interview with The Wall Street Journal published Aug 13, Apple’s head of software engineering, Craig Federighi, attributed a lot of the confusion to poor communication.

“It’s really clear a lot of messages got jumbled pretty badly in terms of how things were understood,” Federighi said in his interview. “We wish that this would’ve come out a little more clearly for everyone because we feel very positive and strongly about what we’re doing.”

Federighi said Apple’s system is protected from being misused through “multiple levels of auditability,” and that he believes the tool advances privacy protections rather than diminishing them. Additionally, he said, because the scanning tools are stored on people’s phones and not on its servers, security researchers and other tech experts will be able to track how they’re used and whether the system is manipulated to do anything more than what it already does.

“If you look at any other cloud service, they currently are scanning photos by looking at every single photo in the cloud and analyzing it; we wanted to be able to spot such photos in the cloud without looking at people’s photos,” he said. “This isn’t doing some analysis for, ‘Did you have a picture of your child in the bathtub?’ Or, for that matter, ‘Did you have a picture of some pornography of any other sort?’ This is literally only matching on the exact fingerprints of specific known child pornographic images.”

He also sought to argue that the scanning feature is separate from Apple’s other plans to alert children about when they’re sending or receiving explicit images in the company’s Messages app for SMS or iMessage. In that case, Apple said, it’s focused on educating parents and children, and isn’t scanning those images against its database of child abuse images.