Part 1 — Kalota’s Primer
Before reading Kalota’s primer, I believed that AI always provided accurate and reliable information. Since it had been a few years since I was last in school, starting this MSW program was my first real exposure to using AI tools in an academic setting. Some of my friends even told me that I could even use AI to do my assignments for me, and that it could handle everything on its own. This made me think that AI must have access to all the correct information, that it scans every article online and always gives the right answer. However, after reading Kalota’s article, I realized I was wrong. I learned that AI does not actually know facts, and as Dr. P mentioned in class, it cannot provide truly human-like responses.
I’ve already used generative AI without really understanding how it worked. For example, when I needed facts or accurate information, the first thing I would do was use AI tools to find it and accept the answers it gives without verifying them through research. I also asked AI chatbots for advice about mental health, not realizing that the information they provided hadn’t been checked for accuracy.
if a client or colleague asked me why AI sometimes “hallucinates” or confidently gives wrong answers, I would explain that AI doesn’t know things the way humans do. Instead, it generates responses by predicting which words are likely to come next, based on patterns it learned from large amounts of data. Sometimes those predictions are wrong, but the AI still sounds confident because it isn’t aware of its mistakes, it’s just following statistical patterns, not understanding or fact-checking like a person would.
Part 2 — Managing, Not Just Rejecting, AI in Mental-Health-Adjacent Practice
If an agency wanted to use AI in mental health for lower-stakes tasks such as intake screening, waitlist triage, psychoeducation, mindfulness training, or after-hours support, I think its use could be acceptable, but only with strict limitations. One key safeguard would be requiring the app to clearly state that responses are generated by AI. Additionally, any AI-generated response that could impact client care should be reviewed by a qualified mental health professional before being shared. There should also be evidence that the AI tool is safe and effective before it’s widely used. These measures would help catch and correct responses that might reinforce stigma, provide harmful advice, or misinterpret a client’s needs.
Moore et al.’s findings helped me draw a clear line between appropriate uses for AI and tasks that require a human. AI can be helpful for basic administrative tasks, like initial check-ins or sorting waitlists, as long as it doesn’t provide detailed information or advice. However, any task that requires understanding a client’s situation, emotions, or needs must be handled by a human professional. The examples in the study of AI giving biased or inappropriate responses reinforced for me that technology cannot replace the essential human qualities in social work.
In my experience, I’ve seen what happens when oversight is missing, not just with AI, but with automated systems in general. For example, in my workplace, we use an automated scheduling system. Many clients are not good with technology and assume that clicking on one service will guarantee them the support they need. Sometimes, they arrive for an appointment only to find they’ve been scheduled with the wrong provider, and we have to reschedule them, which can be frustrating. This experience showed me how easily mistakes can happen without human checking and how important it is to have real people involved to ensure clients get the right care.
Cho,
You have a good start to understanding AI and what it knows and doesn’t know. It has been trained on data, but that data is voluminous and all-encompassing. Some of it is well-sourced, and some of it isn’t. Once its training is complete, it doesn’t “learn.” AI does have amazingly complete and good data. When you ask it something, in most instances it can accurately answer your question. If you ask it to assist you with organizing data you enter, it usually does that well. But it doesn’t reason and think; it predicts based on ALL the data on which it has been trained. Usually that is excellent. But it makes mistakes because sometimes the predictions are incorrect. I have added the How AI Learns document because I know this can be confusing.
You are right on in terms of the questions we have to be able to answer to utilize AI in positive ways that assist our clients. The issue is where we draw the line, and how we make sure there are sufficient safeguards to protect clients and professionals.
I look forward to reading your Final Project to see how you think we might make that happen.
Dr P