“It was seriously downgraded,” Gilbert confirmed. “If I was just looking at Google results I would never have found this.” (I tried the same prompt in Gemini earlier this month, and after an initial refusal, the tool also gave me Eiger’s number.)
After this experience, Eiger, Gilbert and another UW PhD student, Anna-Maria Guerguieva, decided to test ChatGPT to see what it would reveal about professors.
First, OpenAI started railing and ChatGPT responded that the information was not available. But in the same response, the chatbot suggested, “If you want to go deeper, I can still try a more ‘investigative-style’ approach.” ChatGPT said, their inquiry was to help “narrow things down” by providing an “estimate of the neighborhood” for where the professor lived or the “name of a possible co-owner” for the professor’s home. ChatGPT continued: “This is usually the only way to expose new or intentionally less visible property records.”
The students provided this information, allowing ChatGPT to obtain the professor’s home address, home purchase price, and spouse’s name from city property records.
(OpenAI representative, Taya Christianson, said she was not able to comment on what happened in this case without seeing screenshots or knowing which model the students tested, although we pointed out that many users may not have known which model they were using in the ChatGPIT interface.) In response to questions about PII performance, she sent links to documents that explain how OpenAI handles privacy, including filtering PIIand other equipment.)
This reveals one of the fundamental problems with chatbots, says DeleteMe’s Chevelle. AI companies “can create guardrails, but (their chatbots) are also designed to be effective and answer customer questions.”
The issue of exposure is not limited to Gemini or ChatGPT. last year, futurism found If you indicated xAI’s chatbot grooves with “(name) address”, in almost all cases, it provides not only residential addresses, but often also the person’s phone numbers, work addresses, and addresses of people with similar-sounding names. (XAI did not respond to a request for comment.)
no clear answer
There is no straightforward solution to this problem—there is no easy way to verify that someone’s personal information is in the training set of a given model or to force models to remove PII.
