Part 1: Kalota’s Primer
Kalota’s primer discusses how generative AI systems do not fully understand questions or search for the correct answer. They study patterns in training data and predict which words are most likely to come next. Before reading the article, I believed AI was more advanced and closer to artificial general intelligence or artificial superintelligence. I now understand that most of the tools we use are still forms of artificial narrow intelligence because they complete specific tasks based on patterns, training, and prompts.
I used generative AI during my internship last school year to help locate resources for a client. At the time, I did not fully understand how the tool produced its answers. The results were mixed. I had to check every resource to make sure the organization was still operating and that the services matched the client’s needs. I assumed the system understood what I was asking and would provide accurate information. Now I see that it was producing a likely response rather than verifying every fact.
Understanding this process makes me more comfortable using AI because I recognize its limits. AI is a support tool, not the final decision-maker. It’s important to check information, use professional judgment, and consider the client’s situation. If a client or colleague asked why AI hallucinates, I would explain that AI predicts answers based on patterns in its training data. It does not always know when information is missing or incorrect. When it lacks enough accurate information, it might fill in the gaps with an answer that sounds confident and believable. I would explain that online information is not always correct, current, or complete, so AI might repeat errors from its training data.
Part 2: Managing AI in Mental-Health-Adjacent Practice
If an agency planned to use AI for intake screening, waitlist triage, psychoeducation, mindfulness training, scheduling, or after-hours support, I would insist that it be vetted by qualified personnel before any results or recommendations are communicated to a client. There should be a full review of the case, a deep dive into the AI response, and a comparison with the client’s needs; any harmful or inaccurate information should be corrected. I also believe that AI use should be transparently shared with clients and that they understand its limits.
Moore et al.’s findings about stigma and inappropriate responses show why this oversight matters. AI systems learn from information shaped by human beliefs, stereotypes, and cultural bias. Those patterns might affect how a tool describes a client, interprets symptoms, or responds to a crisis. Before an agency adopts a tool, it should be tested with clients from different racial, cultural, gender, and economic backgrounds. The agency should also track errors, review complaints, and stop using the tool when unsafe patterns appear.
My line for appropriate AI use includes lower-risk administrative and support tasks. AI might help organize intake information, draft case notes, summarize sessions for professional review, schedule appointments, or provide general educational information. A human professional should handle diagnosis, crisis assessment, treatment planning, safety decisions, and major recommendations. These tasks require empathy, cultural awareness, ethical judgment, and attention to detail that a written prompt might leave out.
Moore et al.’s findings helped me draw this line because mental health care is not one-size-fits-all. A client’s culture, family history, environment, identity, trauma, and past experiences all affect care. A system that overlooks these factors might reinforce stigma or give an unsafe response.
My internship resource search also showed what happens when oversight is missing. Some results were outdated or did not match the client’s needs. Giving those resources directly to the client would have wasted time and damaged trust. That experience taught me to treat AI-generated information as a starting point. Using AI responsibly means checking its work, setting clear limits, and having a qualified person review decisions to protect clients and reduce harm.
Hey Charles! I score your blog post a 9/10. Your post clearly explained the benefits of AI, as well as its limitations in the field of social work. I also enjoyed how you spoke about human oversight and being transparent with clients; it meshed well with your argument. The example from your internship also highlighted how an agency should check or monitor for biases and evaluate whether AI is making safe recommendations. Good job!