Kalota (2024)

ChatGPT came out when I was in my final year of undergrad, and I had heard of students trying to use it to write papers and assignments for them. I remember many of the professors I had mentioned to us in class that when completing an assignment, refrain from using AI to complete them because they can tell and have tools to check. I was scared to use it because of the cautionary tales of other students, and because of the repercussions that came along with using it. I was not too familiar with AI when it first came out, and did not use it until trends online came out about inputting a certain prompt to ChatGPT to get a picture. I had tried it out then, and in recent years, I have used notebookLM to assist in creating flow charts for different topics discussed in class that I struggled to understand, even when reading materials. I have a better understanding of AI now and try to refrain from using it after learning about the environmental impacts AI has and how AI stores information. I was not familiar with deep learning that ChatGPT uses when first using it to generate images, and after learning about how this tyep of learning “requires a large amount of data for training and prediction” and how it almost mimics the neural structure of the brain (Kalota, 2024), it is in impressive to hear but is also worrying because of how the information I fed it can be stored and used to help the AI interact with others.

When reading about the potential it has to assist people, such as in higher educational settings and in the medical field, it makes me think of AI more positively. Since there is still concern with the security of information, environmental impacts, and the effect on people’s intellectual abilities, I am not fully comfortable with using AI in social work. Client confidentiality and sensitive information are an important part of the work we do, and if I am not 100% sure that a client’s privacy is protected when using AI, I would prefer not to use it in practice.

The way I would try to explain it to them would be that the AI tool that they used is inferring or predicting information to give you based off of its learning process that it used to collect data. The data that the AI found may not be exactly related to what you asked, but based on the patterns it found, it gave you an educated guess. I would also communicate that it is important to check the information it gave you with other professionals, or simple research, so that you are not working with incorrect information.

Moore et al. (2025)

Given the research from this article, I do not feel confident in LLM being used as an alternative form of therapy. If I had to choose a human-oversight point I would insist on to use chatbots, it would be to use an AI that uses supervised machine learning, where the competencies, guidelines, and ethics codes of the professions can be given to the AI to go off of, along with additional training that professionals deem important. Maybe this could be a way to mitigate the stigma that LLM’s use.

I would draw the line at the AI providing any therapeutic intervention or advice to the client. As shown by the results of Moore et al.’s findings, LLM shows stigma and makes harmful or dangerous suggestions to clients when they are experiencing delusions, suicidal ideation, and hallucinations, so this does not give me confidence in using AI in situations where the mental health professional is not present. I think tasks that are suitable for AI in the mental health field would be administrative tasks and data collection. These tools could be used to assist with intaking new clients and assisting with note-taking and documentation.

I think that when human oversight is missing in work settings, assumptions are made and miscommunication happens. Assumptions can be made because of a lack of guidance and information, and can be harmful to the people receiving this communication. The miscommunication can happen because necessary steps are skipped, and then mistakes could be the result of this. Guidance and collaboration are needed in any professional position, whether it be in the form of AI or human-to-human, to avoid mistakes and possibly dangerous outcomes