Crisis safety

AI models can and do give unsafe or wrong guidance in a mental health crisis. Mellow's crisis handling is deliberately constrained so that the highest-risk responses do not depend on free-form model output.

Where screening runs

Mellow does not scan every message in a server. Screening runs on:

• /crisis analyze and /crisis resources
• the "Check on this message" message context menu
• DMs to Mellow
• messages that mention or reply to Mellow, when the author has crisis_detection enabled (the default)

A user can turn screening of their own messages off with /preferences set crisis_detection:false.

Two-stage classification

1. Local screen (no AI). The message is checked against curated keyword and regular-expression lists for direct statements of self-harm, intent, plan, or means. Generic phrases like "help with homework" are excluded. If nothing matches, the result is none and Mellow does nothing special.
2. Constrained classifier (AI). If the local screen matches and the AI is available, a single constrained request classifies the message as none, low, medium, high, or critical, with instructions to be conservative and only use high/critical for a clear expression of intent, plan, or means. The final level is never below medium once the local screen has matched. If the AI is unavailable, a regex-pattern match is treated as high and a keyword-only match as medium.

How Mellow responds by level

The constrained AI calls that are used (the low/medium acknowledgement, and the intro line for /crisis resources) are instructed never to list hotlines themselves, never to give methods or step-by-step advice, and never to ask questions. Their output is length-checked and rejected if it contains method-like content, falling back to fixed copy.

The fixed resource block

Every crisis response ends with a reviewed block, never model output. It is chosen by the user's country (set with /preferences set country: or on the dashboard):

• If you are in immediate danger, call your local emergency number now. For a listed country this names the right number, for example 911, 999, 000, or 112.
• That country's national crisis lines and text services, for example 988 in the US and Canada, 116 123 in the UK and Ireland, 13 11 14 in Australia, 3114 in France, and 024 in Spain.
• Anywhere: findahelpline.com and iasp.info/resources/Crisis_Centres.
• A reminder that Mellow is an AI, not a substitute for a person or professional.

If no country is set, or it is not one of the supported countries, the block falls back to a general one: US lines (988, and text HOME to 741741) plus the international directories. Supported countries are listed under

.

The list is maintained by the team and reviewed by hand. Numbers change, so if you spot one that is wrong or missing, please open an issue.

Moderator alerts

When crisis_alerts is enabled for a server and mod_alert_channel is set, high and critical results post a brief notification (user, level, short summary) to that channel so a human can follow up. The notification does not include the full message content. If no alert channel is configured, /guilddebug will point that out.

Data

Crisis events store an encrypted details field and are kept for safety follow-up. A user can see their own recorded events with /crisis history, and request removal by contacting the team.

Limits

• Mellow is not an emergency service and cannot contact emergency services on anyone's behalf.
• Screening is best-effort. It will miss things and will sometimes over-flag.
• The constrained design trades some warmth and nuance at high/critical for predictability. That is intentional.