Part I – Kalota’s Primer
Kalota’s article provided a lot of useful information regarding how AI works. One concept that changed my perspective of AI is that it creates new information based on training or pattern recognition. I assumed AI to be like a search engine such as Google, that pulls information from various sources, but much faster.
I have encountered generative AI in my personal and professional life. I primarily use ChatGPT to help me stay organized or to brainstorm. I may also use it to ask random questions because it provides information quickly. The tool has been helpful, however I didn’t fully consider how the responses were generated and that AI can sometimes provide inaccurate information. I recently encountered an incident where several people asked the same question on Google and AI Overview provided different inaccurate answers. This encounter as well as the article made me understand the importance of checking reliable sources for factual information
Understanding how AI operates makes me feel uncomfortable with it being used in social work practice. Knowing that AI predicts responses based on pattern recognition means a potential of providing incorrect information. In my experience as a case manager, if AI was used to draft court reports, documentation, case notes, etc., there is a risk that it could leave out important details or include information that does not pertain to the case. This could negatively impact the safety and permanency of a child or family. There’s also a risk of social workers becoming over reliant on AI for recommendations or decision making, hindering their ability to think critically or provide empathy.When explaining to a client why an AI tool “hallucinates” I would say to them that AI is still developing and just like with any other system, it has its flaws. I would also explain that AI systems rely on data that is put into the system and may generate information that is not true.
Part II – Moore et al.
One safeguard that should be put in place to limit reckless AI use is a professional overseeing the system. For an example, an agency who uses AI for after-hours support should have a professional monitoring the application to ensure the safety and well-being of a client. Also, if a client expresses suicidal thoughts or harmful behavior, AI should provide emergency contact information and alert the agency as soon as possible.
In my opinion, AI used being used for intake, to find resources or services, or documentation is appropriate. However, AI being used to make safety/ethical decisions, diagnosis, assessments or anything of that nature is where I would draw the line. AI should be used as a tool to help guide humans (social workers) not used to make decisions that require empathy, understanding, and critical thinking.
As a case manager, using AI to make safety decisions or making reports could negatively impact the children and families being served. In the paragraph above, I stated that AI can be used for documentation, but I have seen how even that can be harmful due to privacy concerns. The agency that I worked for strictly prohibited AI use for anything work related. I can understand why because it poses a risk for inaccurate information or overlooked information, privacy and ethical concerns.
References
Kalota, F. (2024). A Primer on Generative Artificial Intelligence. Education Sciences, 14(2), 172. https://doi.org/10.3390/educsci14020172
Moore, J., Grabb, D., Agnew, W., Klyman, K., Chancellor, S., Ong, D. C., & Haber, N. (2025). Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers. Expressing Stigma and Inappropriate Responses Prevents LLMs From Safely Replacing Mental Health Providers., 599–627. https://doi.org/10.1145/3715275.3732039
I really enjoyed reading your blog post — I’d give it a 10/10 for your convincing arguments. It’s obvious you have serious concerns about the ethical issues around AI and the responses it generates. That’s something I worry about as well, especially knowing that these tools rely on algorithms and can easily spread inaccurate information. As you pointed out, this becomes even more concerning when working with vulnerable populations.You made a strong case for why using AI in social work comes with significant risks. I also appreciated your point about protocols for organizations that use AI for after-hours support, having a professional monitor the application is essential for ensuring client safety and well-being. And if a client expresses suicidal thoughts or harmful behavior, AI should absolutely provide emergency contact information and alert the agency right away. That kind of safeguard is crucial, especially in crisis settings, to make sure quality care is always provided.
Hi! I would rate your post a 10/10. I believe it is very convincing, your stance on AI use within social work when working with clients. Professional oversight will be and is one of the most critical things to consider when incorporating AI into your social work practice. I would even go as far to argue that there should be a separate job position for that specific purpose.
Hello!
Great blog! Score: 10/10
I really enjoyed reading your post because I could relate to your experience. I also thought AI worked like an advanced version of Google before reading Kalota’s article, so it was interesting to see that we had a similar perspective. I agree that AI is much more efficient and can provide information much faster than using a traditional search engine. I remember spending a lot of time searching through Google trying to find exactly what I was looking for, which could become frustrating and overwhelming. One thing I appreciate about AI is that if it doesn’t fully understand my question, it often asks follow-up questions or tries to clarify what I mean before responding. At the same time, I also agree with your point that the information should always be verified, especially in social work, because AI can still provide inaccurate information. Your experience as a case manager also strengthened your argument about why human oversight is essential when working with children and families.
Hey! I would rate your blog post a 10/10. I thought your direct experience as a case manager brought a very convincing perspective to your argument. I also liked how you brought a new perspective by arguing that even using AI for documentation is questionable because of AI’s liability regarding privacy. I also appreciated your point about the potential danger of social workers becoming overly reliant on AI, which could harm their ability to think critically.