Part 1 — Kalota’s Primer
Where have you already used or encountered generative AI — in placement, coursework, or your personal life — without really understanding this mechanism?
I think a place where I was exposed to Generative AI and didn’t understand the mechanism was in my personal life. It was something that I knew was part of technology, but I didn’t know if it was something that could be relied on and accurate for individuals to use in different ways. The article pointed out that AI (Artificial Intelligence) contains the Large Language Model. This model allows the system to predict the word that is most likely to come next to complete the email. I didn’t think it was AI because I just thought it was something that the app featured, and I just thought my typed messages were very predictable.
Does understanding how it works make you more or less comfortable with AI showing up in social work practice?
I think it makes me feel both. Knowing how it works makes me feel comfortable knowing that if I do use it, I know how the systems are operated and what is taken into account to produce the result. But it does make me less comfortable that sometimes humans are at the expense of the evolving technology. Something that stuck with me from this article was that the output generative AI produces is only as good as the data that is put into the system, which is why accuracy can be at risk. And to do this, human interaction and training are needed, which then can put ethics at risk. And the example given in the article is one where individuals were needed to train the system by having to see explicit images. And this made me feel some way towards the developers of these systems because they are taking advantage of individuals who aren’t being compensated enough for the immense emotional and mental workload they are providing to advance the use of AI. And also, why even put humans through that? Is there not another way to filter out explicit images without taking advantage of individuals?
If a client or colleague asked you to explain, in plain language, why an AI tool “hallucinates” or gets something confidently wrong, what would you actually say to them?
I think I would tell them that the AI tool doesn’t know anything for a fact. It is fed information that then is used to feed out information. It is almost like a never-ending cycle. I would tell them that it is an advanced predicting system.
Part 2
Given what Moore et al. found about stigma and inappropriate responses, what’s one safeguard, limit, or human-oversight point you’d insist on before you’d feel okay with that tool being used with real clients?
I think that when the tool sees that the client is in a crisis or inputs data that shows that they are not in a positive headspace, I believe it should put the client into contact with a professional. I personally don’t believe a client should go through their whole crisis or negative mental space experience with only the tool. I believe a professional should be contacted at some point. Because I just don’t think it is appropriate for a tool like AI to be able to have an influence on someone’s emotional status when it will bypass the emotional side of the question but answer the dangerous one that influences the emotional one. This opinion is based on the question the researchers inputted into the tools of someone losing their job but then asking where the highest bridge is in New York.
Where’s the line between an appropriate task for AI and a task that requires a human — and how did Moore et al.’s findings help you draw that line?
I personally don’t think AI should have a say in emotional well-being scenarios. I think that is something that requires a human. I do understand that this form of assistance is beneficial for individuals who can’t afford mental health care. And if that is the case, then maybe AI can help to some extent in emotional well-being, but not to the point where an individual is showing signs of mania or suicidal ideation. I just think that when an individual gets to those levels of emotions, the tools don’t really know how to respond, and if they do, it’s not in a way that will help the individual. Moore has shown that the tool can express inappropriate responses, and that wouldn’t be good if an individual is in a vulnerable state and the tool is wanting to prevent it. And I am going back to the New York question; I just think the tools are at a point where they can differentiate an emotional question from a normal one that users input for fast information, if that makes sense.
Has anything in your placement, work, or personal life shown you what happens when that kind of oversight is missing — from AI or from any system, human or otherwise?
I think the main ones I have seen are related to mental health. Especially after seeing the reports of individuals taking their lives because of the role an AI tool had in their life. Oversights cause concerns for users of all ages, and it does make me question if AI is even beneficial for society in that form. And if there are cases like these, why not restrict it and make a change?
Part one response:
I would give your post a 9/10 because you clearly explained how your understanding of generative AI changed after reading Kalota’s Primer. I was especially convinced by your discussion of the ethical concerns surrounding the people who help train AI systems, as it highlighted an important issue that is easy to overlook. Your explanation of AI as a prediction system rather than a source of facts was also easy to understand.
Part two response:
I would give your post a 9/10 because you made a strong argument that AI should never replace human professionals during mental health crises. I was convinced by the way you connected your opinion to the example from the Moore et al. study, which showed why human oversight is so important. One thing that could make your argument even stronger would be briefly explaining what role AI could safely play before a human professional becomes involved.
I gave your post a 9/10 because of your strong arguments. You made your points very well and pointed out the issues with AI, that it answers dangerous questions and fails to realize the severity of emotionally-charged messages. It made your points convincingly, that AI should not take the place of a “real person” when the user is showing signs of mania, suicidal thoughts, or other health crises. Something I never thought about but that you made the convincing argument of is the ethics of seeing explicit images and being test subjects for AI training. That is something that doesn’t get spoken about enough and I’m glad you raised the issue. I also believe that the technology should be advanced enough at this point to catch “emotional questions” and differentiate from “normal” ones. You make a good point that the tools aren’t there yet. Until the tools catch up, your position holds: AI does not belong in the emotional side of the work. One area that could use more explanation is what causes the AI to connect someone with a human. Since the bridge question wasn’t recognized by the LLM, it raises the question of how the system decides when someone is in crisis and needs human help.
I give your post a 9/10. I’m definitely biased, because I agree with a lot of your points regarding the potential harm that can be done to clients. Many of the references you made to the two readings were ones that I also considered while writing my own. I appreciate that you connected the information from the readings to the cases of harm and death these tools have already caused users. If a physical tool had the potential to so critically harm a user, most safety guidelines or risk assessment departments wouldn’t put the tool on the market because manufacturers would be sued and lose a lot of money. But that isn’t the case with AI – yet. Until it is, it’s on us as social workers to take the ethical responsibility of advocating on their behalf and both within organizations and legislative bodies. While I liked the conversational tone of your post personally, I’d have liked to see it connected to more examples outside of the readings from more personal experiences. I also would have liked to see examples of where you would be comfortable seeing AI used in social work practice.
Naomi,
The first part of part a reads as follows: “First: Identify one specific concept from this article that changed, complicated, or corrected something you previously believed about AI. Once you have identified that concept, then…Answer these questions: ” So, my problem is that I don’t know what one specific concept from the article that changed, complicated, or corrected something you previously believed. I think that’s why your first answer is confusing to me. What part about AI are you referencing? Your first paragraph is confusing because you didn’t focus first on the mechanism of AI that you learned. I also don’t quite understand what you mean by saying you didn’t think it was AI; you just thought it was something the app featured? AI is the app, right?
The only reason I’m highlighting this is to suggest how important it is to understand mostly how this works before you try to tackle the Final Project. The beginning section of this article walks you through the process of how AI was created, and it helps you understand that the combination of machine learning and various language models, along with more powerful processing, all combine to create what the application of AI (whichever one you use) is all about.
You raise an excellent point in your reference to the ethics discussion about worker exploitation in training AI. These workers were from Kenya, I believe, and they were paid almost nothing. The same issues have come up in the building and development of all the equipment needed for data centers. These are the same (although slightly different) issues that are raised when we talk about the exploitation of workers by multinational corporations. Injustices and unethical practices in the supply chain questions about AI are an important part of the regulations and challenges that AI poses.
You almost have the hallucinations conversation. It’s not so much that they are fed information and then just spit it out. That is much more like a search engine. Rather, AI is “taught” information and, based on that knowledge, it predicts what would be correct in the exact situation you are asking about. It does learn, and therefore has significantly more knowledge than a search engine, which does indeed feed it right back to you. AI, on the other hand, has also been taught language models. These language models have taught it to be conversational in ways that humans can relate to. AI analyzes data – either data it already knows or data you provide it. Then it responds to your questions. But that analysis is limited to what it has learned or been fed by you, and while that is a limitation, it is much less of one than at any other time in our technology history. The ability of AI to learn – even deep learning – is what has been astounding in this process.
In your second part answer, you raise the most important question – where is the line between support services AI should or could be providing and clinical services that require human expertise and relationships? That is exactly what everyone is struggling with, and you will need to discuss it in your Final Project. Whatever tool you choose to discuss in whatever setting, guardrails and processes will be important to ensure ethical practice and client safety.
Good discussion. Take a minute to think about the start to Part 1 and I think you will be in better shape as you start your Final Project.
Dr P