Is Mental Therapy a Life-Critical Task?
It would seem reasonable to ban LLMa for life-critical tasks. LLMs are being used for Mental Therapy – for making mentally troubled people feel better about themselves. How is it working out? The New York Times has just published an article which tells us. You can read the NYT essay here . OpenAI is caught in a trap – if they try to prevent mental health problems in their users by making the system less sycophantic, the usage of the system goes down. If they are notified of people who are at risk, the usage goes down. Several problems – 1.The system does not understand what words mean, so they will be looking for trigger words. You can easily write a downer piece without using trigger words. 2. If you go too upbeat when the user is downbeat, the bond between user and Chatbot is broken, “But some users were unhappy with this new, safer model. They said it was colder, and they felt as if they had lost a friend . ” Only a skilled human th...
