愛污传媒

Skip to main content

'Impact on the victims has been devastating': San Francisco goes after websites that make AI deepfake nudes of women and girls

San Francisco City Attorney David Chiu speaks at a public safety town hall meeting in San Francisco on Sept. 18, 2023. (Gabrielle Lurie / San Francisco Chronicle via AP)
San Francisco City Attorney David Chiu speaks at a public safety town hall meeting in San Francisco on Sept. 18, 2023. (Gabrielle Lurie / San Francisco Chronicle via AP)
Share

Nearly a year after AI-generated nude images of high school girls upended a community in southern Spain, a juvenile court this summer sentenced 15 of their classmates to a year of probation.

But the artificial intelligence tool used to create the harmful deepfakes is still easily accessible on the internet, promising to "undress any photo" uploaded to the website within seconds.

Now a new effort to shut down the app and others like it is being pursued in California, where San Francisco this week filed a first-of-its-kind lawsuit that experts say could set a precedent but will also face many hurdles.

"The proliferation of these images has exploited a shocking number of women and girls across the globe," said David Chiu, the elected city attorney of San Francisco who brought the case against a group of widely visited websites tied to entities in California, New Mexico, Estonia, Serbia, the United Kingdom and elsewhere.

"These images are used to bully, humiliate and threaten women and girls," he said in an interview with The Associated Press. "And the impact on the victims has been devastating on their reputation, mental health, loss of autonomy, and in some instances, causing some to become suicidal."

The lawsuit brought on behalf of the people of California alleges that the services broke numerous state laws against fraudulent business practices, non-consensual pornography and the sexual abuse of children. But it can be hard to determine who runs the apps, which are unavailable in phone app stores but still easily found on the internet.

Contacted late last year by the AP, one service claimed by email that its "CEO is based and moves throughout the USA" but declined to provide any evidence or answer other questions. The AP is not naming the specific apps being sued in order to not promote them.

"There are a number of sites where we don't know at this moment exactly who these operators are and where they're operating from, but we have investigative tools and subpoena authority to dig into that," Chiu said. "And we will certainly utilize our powers in the course of this litigation."

Many of the tools are being used to create realistic fakes that "nudify" photos of clothed adult women, including celebrities, without their consent. But they have also popped up in schools around the world, from Australia to Beverly Hills in California, typically with boys creating the images of female classmates that then circulate through social media.

In one of the first widely publicized cases last September in Almendralejo, Spain, a physician who helped bring it to the public's attention after her daughter was among the victims said she is satisfied by the severity of the sentence their classmates are facing after a court decision earlier this summer.

But it is "not only the responsibility of society, of education, of parents and schools, but also the responsibility of the digital giants that profit from all this garbage," Dr. Miriam Al Adib Mendiri said in an interview Friday.

She applauded San Francisco's action but said more efforts are needed, including from bigger companies like California-based Meta Platforms and its subsidiary WhatsApp, which was used to circulate the images in Spain.

While schools and law enforcement agencies have sought to punish those who make and share the deepfakes, authorities have struggled with what to do about the tools themselves.

In January, the executive branch of the European Union explained in a letter to a Spanish member of the European Parliament that the app used in Almendralejo "does not appear" to fall under the bloc's sweeping new rules for bolstering online safety because it is not a big enough platform.

Organizations that have been tracking the growth of AI-generated child sexual abuse material will be closely following the San Francisco case.

The lawsuit "has the potential to set legal precedent in this area," said Emily Slifer, the director of policy at Thorn, an organization that works to combat the sexual exploitation of children.

A researcher at Stanford University said that because so many of the defendants are based outside the U.S., it will be harder to bring them to justice.

Chiu "has an uphill battle with this case, but may be able to get some of the sites taken offline if the defendants running them ignore the lawsuit," said Stanford's Riana Pfefferkorn.

She said that could happen if the city wins by default in their absence and obtains orders affecting domain-name registrars, web hosts and payment processors "that would effectively shutter those sites even if their owners never appear in the litigation."

CTVNews.ca Top Stories

The president and CEO of New Brunswick-based Covered Bridge Potato Chips is taking an 'extended leave of absence' after being charged with domestic violence this past weekend.

A memorial is growing outside a Walmart in Halifax after a 19-year-old employee was found dead inside an oven in the store Saturday night.

A search has started at Prairie Green Landfill for the remains of two victims of a serial killer.

He is a familiar face to residents of a neighbourhood just west of Roncesvalles Avenue.

Canada's discount airline is suspending operations to and from Saskatoon.

A new report suggests that Canadians' exposure to a radioactive gas is increasing, putting millions of people at a higher risk of developing lung cancer.

Local Spotlight

He is a familiar face to residents of a neighbourhood just west of Roncesvalles Avenue.

A meteor lit up our region's sky last night 鈥 with a large fireball shooting across the horizon over Lake Erie at around 7:00 p.m.

Residents of Ottawa's Rideauview neighbourhood say an aggressive wild turkey has become a problem.

A man who lost his life while trying to rescue people from floodwaters, and a 13-year-old boy who saved his family from a dog attack, are among the Nova Scotians who received a medal for bravery Tuesday.

A newly minted Winnipegger is hoping a world record attempt will help bring awareness for the need for more pump track facilities in the city.

A Springfield, Ont. man is being hailed a 'hero' after running into his burning home to save his two infant children.

Hortense Anglin was the oldest graduate to make her way across the platform at York University's Fall Convocation ceremony this week. At the age of 87, she graduated with an Honours degree in Religious Studies.

Looking for a scare with good intentions this Halloween season? The ghosts and ghouls of Eganville, Ont. invite families to tour the Haunted Walk at Lekbor Manor.

The image of a sleepy Saskatchewan small town with 'not a lot going on' is a well-known anecdote. However, one Saskatchewan company is hoping to change that 鈥 and allow communities both on and off the beaten path to share their stories and advertise what they have to offer.