Part 1 — Kalota’s Primer 

In online communications, it can be difficult to tell when someone is using AI to generate content. In the early days it was pretty easy to spot, and sometimes still you might spot a comment reply that doesn’t make sense in the discussion context – but the newer iterations seem to change up their diction patterns and word choices to reflect their users in such a way that it’s less reductive and more flexible.

I would be most comfortable with a SML AI system for client discussion and brain-storming solutions to the issues presented. This way we can be sure the model is trained on competent, evidence-based skills and interventions without the potential harm/dilution of including other methods that haven’t been properly vetted by review boards. I also think I would be comfortable using an AI which could summarize a meeting with a client. This could also save practitioners time in their note-taking, allowing for more cognitive effort to be expended on appropriate mediation and service delivery instead of summarizing (with memory biases) the session that occurred. It also seems imperative that we create a way to implement such a system so that it would be close-looped or off-line to protect against security threats for privacy protections. These I would be more comfortable with compared to some of the predictive client-outcome models discussed in our previous readings.

In describing AI hallucinations, I would emphasize that AI does what many of us typically do already – it searches the internet for what is relevant to the topic and compiles it. A key difference is that people can discern, more or less, whether a source is credible or may have some biases in reporting. Many AI’s do not account for a bias filter or utilize any fact-checking software to determine the accuracy of what it finds. It may find an answer on a forum posted by an uncredentialed user. If stated confidently and often enough in the sources it finds, it becomes regurgitated into the answer as correct. 

Part 2 — Managing, Not Just Rejecting, AI in Mental-Health-Adjacent Practice 

I would like to have a pre-screening system in place so that clients who are in groups known to be stigmatized by LLMs could be directed to limited crisis services or direct human intervention. If a client is not in one such category, those clients could proceed with the tool until a provider was available. However, there should also be limits on how long a client is allowed to be left in the care of an AI provider or oversight of chat logs by moderator AI’s that are programmed to flag for inappropriate responses. I also think some form of warning should be made clear – such as the Surgeon General’s black box labels – for any model that has yet to demonstrate effective safe-guards for responses when dealing with acute distress in patients. Such a measure was considered in 2024 for social media companies, but we still have yet to see that come to fruition. Without some or all of these measures in place, I would feel that my own professional ethics would be at risk.  https://www.npr.org/2024/06/17/nx-s1-5008816/u-s-surgeon-general-calls-for-tobacco-style-warning-labels-for-social-media

For me, the line between what is and isn’t an appropriate task would be one that has reasonable semblance to a human therapy output in a given situation. How does the AI model compare to a group of professionals in responding to certain stimuli? What is the margin of error, and how comparable is it to the error rate of the human group? Moore’s findings of AI responses to certain mental health disorders made it clear how much more work to improve these systems is needed. I would be comfortable with some AI use, such as summarizing client sessions, reviewing chat logs, providing intake questionnaires or handling book-keeping tasks. As Moore stated, there are many therapy-adjacent tasks that may be appropriate to lighten the burden while maintaining client safety.

There was a time when my supervisor was very pro-AI – particularly for writing grants. One aspect she instilled in me was careful review and editing of the responses, and often providing the tool with particular articles and sources so that it didn’t hallucinate any of the information to be included in our bid for funding. Many times feeding the AI source material yielded significantly better results. When pushed for where it got information I hadn’t provided for it, it backed down quickly – usually coming back with a different fact or figure from a specific link that sometimes had little to do with the funding request we were writing. Without careful oversight, we could have easily lost funding that was regularly granted to us for failing to review the AI responses before adding it to the grant draft.

Author’s note: The featured image is from a favorite video game of mine, “Robotherapy” by Lucas Molina, which I thought about repeatedly while writing this post. I think I’ll go back and play it again after this semester is over.