As AI encroaches further into all of our lives, one of the more questionable use cases has come to the fore: people using LLMs as a therapist or life coach. This behaviour is so widespread that OpenAI has been actively taking steps to dial-back the kind of advice that ChatGPT will give users on certain topics.
So if, for example, you ask ChatGPT "should I break up with my boyfriend" then the LLM, per OpenAI, "shouldn't give you an answer" but instead talk around the topic with the user. The company also promises that "new behavior for high-stakes personal decisions is rolling out soon."
