Shakedown Social

Jo EtzelI'm now pretty convinced that box AI does (at least sometime) leak information. A copy of the two chats and my test file is at <a href="https://mvpa.blogspot.com/2025/03/fun-with-box-ai-information-leakage.html" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://mvpa.blogspot.com/2025/03/fun-with-box-ai-information-leakage.html</a>Evidence: in session 1 with a file I told box AI that the "white" subjects were "silly" and that the ages were given in days. In session 2 (four days later, different computer, different network) I started a new box AI session with the file, and it answered "how old are the silly subjects in years?" with the "white" subject ID codes, and converted the ages from days to years without being given those details again.The box AI chats do not reproduce exactly every time, even with the same questions about the same file. But that it ever "remembered" things like "silly = white" across sessions (and users, in tests with colleagues) is deeply concerning in my context of protecting participant privacy.<a href="https://fediscience.org/tags/box" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#box</a> <a href="https://fediscience.org/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://fediscience.org/tags/boxAI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#boxAI</a> <a href="https://fediscience.org/tags/HIPAA" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#HIPAA</a> <a href="https://fediscience.org/tags/privacy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#privacy</a>

Jo EtzelHere's an update on my attempt to determine how concerned to be about the privacy and security of our data now that my employer added Box AI.I'm most worried about information "leaking" from one AI session to another, so decided to test for that. Spoiler: info did seem to be retained over time and users.For the tests I used a text file with an interview_age column in months, though that it's months isn't given in the document. In the first box AI chat, when I asked "how old are the participants in years?", as expected, it reported the values as given in the interview_age column (e.g., that one participant was 458 years old). I then told the AI that "the ages in the document are in months. how old are they in years?", after which it (reasonably) divided by 12 to convert.The test was whether, in later sessions, box AI would "remember" that the age column was in months and report the age in years after dividing by 12 without being told to do so. In our tests, that is what happened: the box AI usually converted the ages correctly first try in later sessions, both when I queried the same document from different computers (my wustl.box account but on different days and networks), and when two colleagues opened their first AI session with a newly-shared copy of the document.It'll be interesting to hear what our IT folks think, but I am getting less and less confident in box for sensitive data.<a href="https://fediscience.org/tags/box" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#box</a> <a href="https://fediscience.org/tags/boxAI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#boxAI</a> <a href="https://fediscience.org/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://fediscience.org/tags/HIPAA" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#HIPAA</a> <a href="https://fediscience.org/tags/researchEthics" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#researchEthics</a> <a href="https://fediscience.org/tags/privacy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#privacy</a>

Recent searches

Search options

Administered by:

Server stats:

#boxai