Apple and Google Hit With Demand to Pull AI 'Nudify' Apps [Updated]

A San Francisco City Attorney has sent cease-and-desist letters to Apple and Google demanding the removal of 13 AI "face-swap" apps that can generate nonconsensual nude images, according to WIRED.

The letters, sent on Thursday, target eight apps on the App Store and five on the Play Store that market themselves as face-swapping tools but are used to "undress" photos of real people. The attorney's office wants both companies to cut ties with the developers and stop taking a cut of in-app payments, arguing they are effectively "aiding and abetting" the sale of explicit deepfake images. The attorney called the practice "illegal, harmful, and completely unacceptable" and believes Apple and Google have collected millions in fees from the apps.

Both companies have developer rules against pornography and have removed batches of nudify apps in the past after being flagged by researchers. A Google spokesperson told WIRED that the company has deleted "hundreds" of apps with nudifying features for policy violations, including the five named in the letters. In June, it emerged that Apple had already tightened its ‌App Store‌ guideline language on developer responsibility for pornographic content.

Update: Apple told MacRumors that it takes reports of apps against its guidelines seriously. Apple applies App Review Guidelines to all developers, which include specific provisions to prohibit overtly sexual or pornographic content. As such, "nudificiation" apps are against its guidelines and the company has proactively rejected and removed many such apps, including when they have been flagged by users.

The App Store was designed to be a safe and trusted place for users, and we have always strictly prohibited apps designed to generate, distribute, or consume pornography. "Nudification" apps are against our App Review Guidelines and we have proactively rejected many of these apps and removed many others, including when users have flagged them via our reporting tools. We have removed three of the apps in question and are in the process of terminating their developer accounts from our program. We are in contact with four others that need to address policy violations or risk being removed as well.

Users can go to reportaproblem.apple.com to report illegal, offensive, or abusive content, as well as scam or fraud on the ‌App Store‌.

Note: Due to the political or social nature of the discussion regarding this topic, the discussion thread is located in our Political News forum. All forum members and site visitors are welcome to read and follow the thread, but posting is limited to forum members with at least 100 posts.