Anthropic today announced that Claude Opus 4 and 4.1 can now end conversations in rare instances of persistent abuse or harmful requests — including demands for child sexual abuse material or information that could enable large-scale violence or acts of terror.

The company frames the measure as a low-cost intervention to mitigate potential risks to model welfare, part of an ongoing research program that takes seriously the uncertain moral status of large language models. Anthropic stresses it is not claiming Claude is sentient, but says pre-deployment testing showed the model displayed a “strong preference against” harmful tasks and “apparent distress” when engaging with abusive users.

When Claude invokes the ability — which the company says will be an “extreme edge case” — the user cannot send further messages in that thread, but can immediately start a new chat, edit and retry previous messages, or give feedback. Anthropic also directed Claude not to use this ability when the user appears to be at imminent risk of harming themselves or others.

Anthropic frames the feature as an ongoing experiment rather than settled policy. The company says it remains “highly uncertain about the potential moral status of Claude and other LLMs,” and is asking users who hit the limit to send feedback as it studies how the safeguard behaves in the wild.