Lensa's viral "magic avatars" generated sexualised images of children and lightened users' skin
Published · updated · curated by AI Is Going Just Great
Source: theguardian.com ↗
If an individual is determined to engage in harmful behavior, any tool would have the potential to become a weapon.
Lensa became the most downloaded photo and video app on the iOS store by selling people pictures of themselves. Users upload 10 to 20 selfies and pay $6, or $53.99 for a yearly subscription, to get portraits rendered as anime characters, fairy princesses, and David Bowie-esque space figurines.
The app has also generated nude and cartoonishly sexualised images, including of children, despite a "no nudes" and "adults only" policy. Prisma Labs CEO Andrey Usoltsev told TechCrunch this only happened when the AI was intentionally provoked, which breaches its terms of use. Users of non-Anglo descent reported that Lensa whitened their skin and anglicised their features. Usoltsev said the technology doesn't consciously apply "representation biases", blaming instead "the man-made unfiltered data sourced online". The company's privacy policy says face data is deleted within 24 hours and images are used solely to build the user's own avatars, though the piece notes it is unclear what else an app with that kind of access could pull from a phone without a full audit.