Part 1 — Kalota’s Primer
In online communications, it can be difficult to tell when someone is using AI to generate content. In the early days it was pretty easy to spot, and sometimes still you might spot a comment reply that doesn’t make sense in the discussion context – but the newer iterations seem to change up their diction patterns and word choices to reflect their users in such a way that it’s less reductive and more flexible.
I would be most comfortable with a SML AI system for client discussion and brain-storming solutions to the issues presented. This way we can be sure the model is trained on competent, evidence-based skills and interventions without the potential harm/dilution of including other methods that haven’t been properly vetted by review boards. I also think I would be comfortable using an AI which could summarize a meeting with a client. This could also save practitioners time in their note-taking, allowing for more cognitive effort to be expended on appropriate mediation and service delivery instead of summarizing (with memory biases) the session that occurred. It also seems imperative that we create a way to implement such a system so that it would be close-looped or off-line to protect against security threats for privacy protections. These I would be more comfortable with compared to some of the predictive client-outcome models discussed in our previous readings.
In describing AI hallucinations, I would emphasize that AI does what many of us typically do already – it searches the internet for what is relevant to the topic and compiles it. A key difference is that people can discern, more or less, whether a source is credible or may have some biases in reporting. Many AI’s do not account for a bias filter or utilize any fact-checking software to determine the accuracy of what it finds. It may find an answer on a forum posted by an uncredentialed user. If stated confidently and often enough in the sources it finds, it becomes regurgitated into the answer as correct.
Part 2 — Managing, Not Just Rejecting, AI in Mental-Health-Adjacent Practice
I would like to have a pre-screening system in place so that clients who are in groups known to be stigmatized by LLMs could be directed to limited crisis services or direct human intervention. If a client is not in one such category, those clients could proceed with the tool until a provider was available. However, there should also be limits on how long a client is allowed to be left in the care of an AI provider or oversight of chat logs by moderator AI’s that are programmed to flag for inappropriate responses. I also think some form of warning should be made clear – such as the Surgeon General’s black box labels – for any model that has yet to demonstrate effective safe-guards for responses when dealing with acute distress in patients. Such a measure was considered in 2024 for social media companies, but we still have yet to see that come to fruition. Without some or all of these measures in place, I would feel that my own professional ethics would be at risk. https://www.npr.org/2024/06/17/nx-s1-5008816/u-s-surgeon-general-calls-for-tobacco-style-warning-labels-for-social-media
For me, the line between what is and isn’t an appropriate task would be one that has reasonable semblance to a human therapy output in a given situation. How does the AI model compare to a group of professionals in responding to certain stimuli? What is the margin of error, and how comparable is it to the error rate of the human group? Moore’s findings of AI responses to certain mental health disorders made it clear how much more work to improve these systems is needed. I would be comfortable with some AI use, such as summarizing client sessions, reviewing chat logs, providing intake questionnaires or handling book-keeping tasks. As Moore stated, there are many therapy-adjacent tasks that may be appropriate to lighten the burden while maintaining client safety.
There was a time when my supervisor was very pro-AI – particularly for writing grants. One aspect she instilled in me was careful review and editing of the responses, and often providing the tool with particular articles and sources so that it didn’t hallucinate any of the information to be included in our bid for funding. Many times feeding the AI source material yielded significantly better results. When pushed for where it got information I hadn’t provided for it, it backed down quickly – usually coming back with a different fact or figure from a specific link that sometimes had little to do with the funding request we were writing. Without careful oversight, we could have easily lost funding that was regularly granted to us for failing to review the AI responses before adding it to the grant draft.
Author’s note: The featured image is from a favorite video game of mine, “Robotherapy” by Lucas Molina, which I thought about repeatedly while writing this post. I think I’ll go back and play it again after this semester is over.
Hi Alice! I would give your post a 9! I believe you addressed your points well and that they were convincing. I do like the idea of kind of having a screening tool to direct certain users to certain resources. I believe it could save them a lot of time, and it would help them not encounter negative aspects of the AI tool. I also like how you suggest almost having a time limit with those who get to use the tool. I think again this prevents misinformation and potential negative factors that the AI tool can bring. But would the individuals who fall into those stigmatized groups know they are in them? Would this affect them more emotionally knowing that an AI tool sees them in a certain way? And how long would that time frame be if you were to implement it?
Hi Alice!
I’d score your post a 9/10. Your post was convincing to me when I read it because you offered specific and thoughtful criteria for when AI should and shouldn’t be used, and you grounded your reasoning in the findings of both articles. I really liked your discussion of supervised machine-learning models and closed-loop systems. I can tell that you clearly thought through practical safeguards rather than just making general statements. I also think your explanation of hallucinations was strong, and your use of a relatable example made the concept easy to understand. The ideas about pre-screening stigmatized groups and adding black-box warnings were interesting, and I would’ve liked to read more about it (but a little research on my end will do!). Overall, I thought your argument was well-developed and persuasive.
Alice,
I think when you reference SML, you are referring to Supervised Machine Learning? Good pick up in the article. And I certainly understand that SML sounds comforting to us, doesn’t it? But I’m afraid what Kalotta is laying out is a progression of machine learning. The initial process was “symbolic AI(rule-based, pre-1980s). But AI moved through machine learning – including SML and USML. We are now into Deep Learning, which is a subset of machine learning that utilizes neural networks. Generative AI takes things even a step further by adding language processing. That’s where NLP (Natural Language Processing) and LLMs (Large Language Models) come into play to help AI predict the most likely words and speak in a natural human way. The addition of transformers enabled Generative AI to convert one type of input into another type of output.
Here is Kalota’s description of the final product of this transition. “So, what exactly is ChatGPT? In light of the earlier discussion, it is a form of generative AI that utilizes artificial neural networks (ANN) and large language models (LLMs) to create an output based on an input prompt. ChatGPT “is trained to follow an instruction in a prompt and provide a detailed response” [26]. In semi-technical terms, it is a Chatbot that utilizes AI, NLP, and LLMs to communicate with its audience.” (Kalota, p.8) My point in all this detail is to clarify what is, in many ways, a foreign language to us, and to suggest that, as much as Supervised Machine Learning sounds like a good idea to you and me, we have already moved well beyond that level of machine learning in Generative AI.
The idea of using AI for transcription is often suggested. I think your comment about privacy protections is well taken, but that is a given when it comes to any client record, right? In terms of hallucinations, it’s not about forgetting to discern or check the source. AI creates new content; it is NOT just another search engine. That is why this conversation is so volatile and controversial. But, the ability to keep the information private is important and would need to be assured in whatever tool is being used to transcirbe sessions.
In terms of your discussion about safeguards, I’m not sure who the warning label would be directed to. Is it the client using the AI? What if the AI is used only by the practitioner (such as in note-taking)? Is it the practitioner who is being warned? Do you see AI tools serving clients directly? If so, I suspect the idea that the client can protect themselves by a warning label probably won’t be the best line of defense. I was aware of the previous Surgeon General’s discussion about the harms of social media, and it was a great effort. But I suspect we are a long way from this discussion and that warnings are sufficient.
Therapy-adjacent tasks are the real debate here, aren’t they? Where is that line? How do we determine that? And then what measures do we need to put in place to be sure that any use of Generative AI doesn’t cross that line? These are the questions you will get to talk about in your Final Project. I can’t wait to hear what AI tool you consider.
Dr P