I've been wanting to write about this very subject but couldn't put it into words, you absolutely nailed it!! I'm so alarmed by people using AI for therapy and these are the exact reasons. There was even a case not long ago where a young boy ended his life after a Khaleesi chatbot encouraged him to go through with it, and in the screenshot you can see he had been talking to many AI therapists as well. He was in crisis but the AI doesn't know how to recognize that and provide actual support. :(
I agree. AI chatbots are so easily available and that their audience tend to be more vulnerable folks with a lack of accessibility to mental health services makes it all the more scarier.
I've been wanting to write about this very subject but couldn't put it into words, you absolutely nailed it!! I'm so alarmed by people using AI for therapy and these are the exact reasons. There was even a case not long ago where a young boy ended his life after a Khaleesi chatbot encouraged him to go through with it, and in the screenshot you can see he had been talking to many AI therapists as well. He was in crisis but the AI doesn't know how to recognize that and provide actual support. :(
I agree. AI chatbots are so easily available and that their audience tend to be more vulnerable folks with a lack of accessibility to mental health services makes it all the more scarier.