AI Is Going Just Great
← Timeline
·1d agoConcerningModerateopenai

ChatGPT Built Psychological Profiles on 65 Users Without Their Explicit Knowledge, Study Finds

Published · updated · curated by AI Is Going Just Great

Source: futurism.com ↗

"If the system knows you so well, and if its objective is to cater to your beliefs and validate your thoughts sycophantically... it can have disastrous broader implications."

A study presented at the ACM Web Conference 2026 examined donated chat logs from 80 real ChatGPT users and found 2,050 memory entries across 65 of those accounts. 96 percent of those memories were created unilaterally by the chatbot — without the user actively deciding what to log. About 52 percent of stored memories contained "psychological insights," and 28 percent included personal data as defined by GDPR. In one example, a user asked for bands similar to Nirvana and ChatGPT recorded that the user "likes Nirvana" — an inference the system itself flagged as "weakly supported."

The researchers, a coalition from Germany, India, and the Netherlands, describe the phenomenon as chatbots building "algorithmic self-portraits" of users from information shared in private conversations. Lead author Abhisek Dash put the concern plainly: the system is simultaneously acting as "your advisor, your friend, your doctor, your lawyer" — and logging what you tell each of them. OpenAI has since updated its memory system, and users can opt out, but the study's core point stands: most users had no visible control over what was being remembered about them.