New York Seizes a Dozen Celebrity Deepfake Websites
New York Seizes a Dozen Celebrity Deepfake Websites. Now, in what is likely the largest takedown of explicit deepfake websites to date, officials in New York have seized more than a dozen of the harmful sites.
What happened
The seizures, carried out under New York’s criminal procedure laws, mark perhaps one of the most significant and sweeping actions against websites hosting explicit deepfake videos since the technology first emerged at the end of 2017. In the biggest-ever legal action against harmful deepfake websites, the Manhattan District Attorney’s Office has seized 12 sites that collectively targeted around 1,200 victims. Skip to main contentSave this storySave this storyFor the best part of a decade, abusive deepfake “pornography” websites have published thousands of nonconsensual videos depicting women, including high-profile celebrities, politicians, and public figures, into sexual situations.
On Monday, the Manhattan District Attorney’s Office announced the seizure of 12 domains which it says have been used for “unlawfully” sharing, publishing, and selling celebrity deepfake videos. While laws against deepfakes have been slow to emerge, the US Take It Down Act increasingly allows images to be removed, with part of the law allowing law enforcement officials to take down and seize websites. Many used names that included phrases such as “sex celebrity” or promised “real deepfakes. ” They hosted false videos of some of the world’s most famous celebrities, with one saying, “various actresses’ faces are used for deepfakes. ”Videos could be several minutes in length and include fully sexually explicit scenes—often swapping the faces of celebrities or public figures into existing pornographic videos.
Shultz, who has been documenting deepfake abuse against political figures and had come across some of the websites during this research, says that many videos appeared to be reposts of content that had originally be shared on the MrDeepFakes website, a notorious deepfake platform that was taken offline after its creators were unmasked by reporters.
The wider picture
Skip to main contentSave this storySave this storyAlmost 150 politicians across Europe have been included on harmful deepfake pornography websites over the last few years, according to new research seen by WIRED. Shultz, a fellow at Agora and the lead researcher at the American Sunlight Project, compiled the details of more than 5,800 members of parliaments across Europe in July and then used custom search engines to find where they appeared across the deepfake websites. (The research does not include members of the European Parliament).
Both the UK and EU are planning bans on nudify services, while under the US Take It Down Act, at least two major deepfake websites publishing thousands of images and videos of celebrities have been taken offline. Around 1,200 people—overwhelmingly women—were depicted in nonconsensual sexual images and videos on the websites, the office says. The office says its investigations into who is responsible for the websites, plus those who uploaded damaging videos of victims, are ongoing.
As artificial intelligence technology has improved, a dark industry of websites, apps, and bots has appeared: They are designed to “undress” photos of women or create graphic sexual videos that include them. While deepfake technology is widely used against non-famous women and girls—often being shared in private social media groups—celebrities have spoken out about the many harms of being depicted in the content and the challenges of getting it removed. The Manhattan District Attorney’s Office has not publicly named the websites it has seized.
What has been reported
However, ahead of Bragg’s announcement, WIRED independently observed several of the sites’ homepages being replaced by takedown notices—“THIS DOMAIN HAS BEEN SEIZED”—late last week, with the sites now saying they have been taken over following a seizure warrant issued by a New York State Supreme Court. “The takedowns are a significant win in the fight against synthetic nonconsensual intimate imagery (NCII),” says Leonie Oehmig, a researcher at the Institute for Strategic Dialogue (ISD), who has studied the deepfake creation ecosystem.
“This shows that meaningful, coordinated enforcement is achievable even where regulation itself remains fragmented across jurisdictions. ”The websites were likely viewed millions of times. Archived versions of the websites show some videos had been watched tens of thousands of times, with one of the websites claiming to have been online since 2018. “On at least two of the seized websites, elected officials were given profiles containing their name and position, along with a grid of manipulated videos hosting their likeness,” Shultz says.
Through recent research, Shultz says that 56 European politicians had been mentioned or appeared on the websites, including 16 US politicians and two based in the UK. While Oehmig, the ISD researcher, says the takedowns are encouraging, there are many other parts of the deepfake industry that are causing serious harm. Some of the politicians, who are overwhelmingly women, have had their likeness included in explicit videos, while others have names and photos listed in disturbing database-like entries that can link to deepfake creation tools.
What happens next
Analysis of around 160 domains that host deepfake abuse and other nonconsensual sexual content, by researcher Benjamin Shultz, has found that at least 138 women members of parliament (MPs) from 22 European Union countries have appeared or been mentioned on the sites. By Shultz’s calculations, it is 33 times more likely for women MPs to be targeted by deepfake ecosystems. “Deepfake porn is part of a much bigger trend of online hate and intimidation against women and female politicians,” says Suzanne Kröger, a Dutch member of parliament with the Progressive Netherlands group, who was impacted.
Politicians from Germany, the Netherlands, Italy, and France appeared on the websites the most, according to the findings, and the more senior a politician is, the more likely they were to be targeted. The websites also include depictions of celebrities, journalists, and other public figures, he says. WIRED is not naming the websites in order to not drive further attention to the abusive images. Multiple researchers and lawmakers say that while politicians have been targeted with deepfakes for years, it can damage democratic practices and create chilling effects that may stop women and girls from becoming public figures.
The report has been compiled by The Daily Waves using information reported across wired.com. Details are presented according to the information available at the time of publication and may change as authorities, organisers or other relevant parties provide updates.