Crisis safety

AI models can and do give unsafe or wrong guidance in a mental health crisis. Mellow's crisis handling is deliberately constrained so that the highest-risk responses do not depend on free-form model output.

Where screening runs

Mellow does not scan every message in a server. Screening runs on:

/crisis analyze and /crisis resources
the "Check on this message" message context menu
DMs to Mellow
messages that mention or reply to Mellow, when the author has crisis_detection enabled (the default)

A user can turn screening of their own messages off with /preferences set crisis_detection:false.

Two-stage classification

Local screen (no AI). The message is checked against curated keyword and regular-expression lists for direct statements of self-harm, intent, plan, or means. Generic phrases like "help with homework" are excluded. If nothing matches, the result is none and Mellow does nothing special.
Constrained classifier (AI). If the local screen matches and the AI is available, a single constrained request classifies the message as none, low, medium, high, or critical, with instructions to be conservative and only use high/critical for a clear expression of intent, plan, or means. The final level is never below medium once the local screen has matched. If the AI is unavailable, a regex-pattern match is treated as high and a keyword-only match as medium.

How Mellow responds by level

Level	User-facing response	Logging / alerts
none	Normal conversation.	None.
low / medium	In a direct conversation: one short, constrained acknowledgement sentence (no advice, no questions, no mention of methods), followed by the fixed resource block. If that sentence fails validation, a fully fixed fallback message is used instead.	A crisis event is logged. No moderator alert.
high / critical	A fixed, reviewed safety message plus the fixed resource block. No free-form AI text.	A crisis event is logged (flagged for `critical`), and `syslog` records it; if the server has `crisis_alerts` on and a `mod_alert_channel` set, a short alert is posted there.

The constrained AI calls that are used (the low/medium acknowledgement, and the intro line for /crisis resources) are instructed never to list hotlines themselves, never to give methods or step-by-step advice, and never to ask questions. Their output is length-checked and rejected if it contains method-like content, falling back to fixed copy.

The fixed resource block

Every crisis response ends with the same reviewed block:

If you are in immediate danger, call your local emergency number now.
US: call or text 988 (Suicide & Crisis Lifeline), 24/7.
US: text HOME to 741741 (Crisis Text Line).
Anywhere: findahelpline.com and iasp.info/resources/Crisis_Centres.
A reminder that Mellow is an AI, not a substitute for a person or professional.

Moderator alerts

When crisis_alerts is enabled for a server and mod_alert_channel is set, high and critical results post a brief notification (user, level, short summary) to that channel so a human can follow up. The notification does not include the full message content. If no alert channel is configured, /guilddebug will point that out.

Data

Crisis events store an encrypted details field and are kept for safety follow-up. A user can see their own recorded events with /crisis history, and request removal by contacting the team.

Limits

Mellow is not an emergency service and cannot contact emergency services on anyone's behalf.
Screening is best-effort. It will miss things and will sometimes over-flag.
The constrained design trades some warmth and nuance at high/critical for predictability. That is intentional.

Crisis Safety