Part 1: Kalota’s Primer
One concept from Kalota’s article that shifted my understanding of generative AI was the explaining of how large language models generate text using probabilities rather than meaning. Kalota writes that a language model “predict what words are likely to come next in a text,” based on statistical patterns rather than comprehension (Kalota, 2024, p.7). This new framing of AI being something that predicts and not as something that knows corrected a misconception I had that I didn’t realize. I knew that AI wasn’t human, but I suppose I continued to assume that there was a deeper understanding happening with AI models.
Once I understood that generative AI is basically a massive pattern-matching system build on training data, I realized how often I’ve used it without grasping the concept. In my life, I have used AI tools to brainstorm and organize my thoughts for school assignments, or even drafting outreach messages with my supervisor in practicum. I treated AI like a smart assistant, but I did not process that it was generating responses by guessing the next most likely word. It makes more sense to me now, the confidence behind the responses even when they’re wrong. Understanding this now makes me want to be more cautious about how I utilize AI, especially in practice. Not that I am completely opposed to its help, but I want to be more aware of its limits.
The article states that generative models ” produce a sequence of words, code, or other data based on an input via the prompt,” meaning that these tools will always produce something, even if they correct answer is unclear (Kalota, 2024, p.7). I think that this can be harmful to clients in a field like ours where accuracy matters. When explaining why AI hallucinates in plain language, I’d say that AI doesn’t understand real world experiences, it predicts text. When AI doesn’t have the right information, it will still try to guess the next likely words, and sometimes those guesses might sound confident but can actually be wrong. For me, learning this doesn’t make me completely against AI, but it does make me more careful and intentional about how I use it.
Part 2: Managing, Not Just Rejecting, AI in Mental-Health-Adjacent Practice
The findings in this article proved that LLMs sometimes reproduce harmful stereotypes or offered inappropriate advice, which are the worries many professionals carry with the new integration of AI. If an agency wanted to use AI for something like intake screenings or psychoeducation, the first safeguard I would insist on is requiring humans to check AI-generated content before it is given to a client. AI can easily help gather information or draft educational material, but a professional check of accuracy, tone, and potential harm would be the most ethical thing to do. Moore et al.’s findings demonstrated that AI can misinterpret emotional content or respond in stigmatizing ways. Therefore, a human quality check shouldn’t be optional, it should be a safety net.
For me, the line between appropriate and inappropriate tasks is that AI can support tasks that are structured or informational, but humans are required for anything relational, interpretive, or emotionally complex. What the article demonstrated helped clarify that distinction for me. If a tool can cause harm by misunderstanding someone’s distress, then it shouldn’t be anywhere near crisis support. My undergrad placement showed me what happens when oversight is missing. As I mentioned in my previous blog post, at the free clinic, clients sometimes received human automated messages that were confusing, incorrect, or anxiety-provoking. I wanted to touch on this experience once again, because it’s a good example of how no one thoroughly reviewed these messages before they were set in stone. When clients are already vulnerable, small errors like this can increase their stress. Thinking back on that field experience makes me wonder what the consequences could have been if an AI system had generated those messages without proper human review.
9/10
Across both posts, you present a clear, thoughtful, and well‑supported position that feels grounded in your actual experiences as a social work student and intern. What makes your writing convincing is the way you connect the readings to your own misconceptions, your practicum experiences, and your developing professional judgment. You don’t just repeat what Kalota or Moore et al. said you show how their points changed your thinking and why those changes matter for your future practice.
You also draw strong boundaries around what AI can and cannot safely do in social‑work‑adjacent settings. Your insistence on human oversight is justified with concrete examples, especially the clinic messaging issue, which illustrates how even small errors can harm vulnerable clients. That real‑world connection strengthens your argument and makes your caution feel practical rather than fearful.
Kailey, I’d give your post a 10/10. Your honest reflection about your previous understanding of AI, your use of it, and how your perspective has grown as a result of these readings was appreciated and helped to highlight a lot of the important aspects of both readings. That you tied that shift in your knowledge-base back to your previous internship experience made it stand out even more. Throughout your post, you incorporated some key take-aways from both of the readings which influenced this shift in your perspective and notated them accurately. You further utilize your internship experience as a grounded, real-world application as to why human oversight is important from an ethical standpoint when working with any sort of system automation – whether it’s AI or older architectures.
I give your post a 10/10. I think your “line” was particularly well articulated, and I also agree. Because of the way that Gen-AI uses LLM to “predict” language instead of true reasoning it should not be used in relational or emotional, or nuanced situations where inference and tone needs to be interpreted. I also used to treat Gen-AI as an “assistant” of sorts, and I still do in some cases, but also like you after reading these articles, I am starting to understand how Gen-AI “reasons”, which has changed my perspective on how I use it and what I expect from it.
Kailey,
It is unnerving to realize that AI is predicting rather than knowing, isn’t it? It is important to remember, however, that AI’s predictions are different than anything we have ever seen before. The predictions with AI involve deep machine learning, and AI is using tremendous data bases to put together the information. You are correct, understanding this fact does change the way we use it and the way we check the answers. When I use AI for any information, I will always check the source. I have learned that several sources didn’t exist.
Now, that all being said, there is an amazing capacity for AI to organize and summarize information, and then provide information to us. Respecting this process requires us to both respect the technology and be cautious with it, right? I once heard a commentator say that with AI we should be curious, but cautious. As for hallucinations, I think it is interesting that AI MUST answer the question, even when it can’t find data to build a reasonable answer. That is an interesting characteristic of this technology.
I believe you are 100% correct that all responses that AI gives an any clinically-=adjacent service should be screened and tested. The issue is how exactly do we do that? If we are saying that everything must be screened in real time, in the moment it is provided, then AI doesn’t offer much to us, does it? When you begin to tackle your Final Project, remember that part of the assignment is to discuss HOW such monitoring will occur and the steps and techniques we ill need to use to be sure we make that happen.
Good discussion.
Dr P
Kailey,
It is true that the idea that AI is predicting instead of knowing is somewhat daunting. But I do believe we have to redefine those words when we are talking about AI. If you read carefully about what machine learning is all about, you can see that these “predictions” are significantly more advanced than a typical search engine. The amount of information AI uses to make these predictions is why Artificial Intelligence provides such good information in most circumstances. That is also why, if you provide it with initial information and use focused prompts, you will get even better results. None of this discounts the fact that it does hallucinate, can be wrong, and results should be checked carefully. That is also what makes the use of AI in clinical settings so complicated.
It sounds like you have a good example of some of the challenges from your work at the free clinic. When you think about “requiring humans to check AI-generated content before it is given to a client,” are you saying everything AI says each time they say it, general responses to general questions, or a constant human review of AI as it is working? Can we create responses and guardrails that would protect clients and let AI do the work? You can see how a human reviewing every AI comment would suggest that we just don’t use AI, right? These are really good ideas and raise really important questions. In your Final Project, remember to talk about what kinds of protections make sense, where they are needed, and how they would be implemented – or some ideas about how.
Good discussion.
Dr P