Why Administrative Friction Prevents Effective Crisis Safety Features

Original Title: ChatGPT created a feature to help people in crisis. But there are some challenges.

The Friction Gap: Why Safety Features Fail in Crisis

OpenAI’s Trusted Contact feature shows a disconnect between product design and human behavior during emergencies. While the feature aims to connect digital interaction with real-world support, it relies on a user-initiated model that clashes with the cognitive state of someone in crisis. When safety features require administrative work like navigating settings, opting in, and pre-configuring contacts, they exclude the people they are meant to protect. For product leaders, this is a warning: a solution that makes sense in a stable state often becomes a barrier during a crisis. Understanding this friction gap is necessary for building systems that provide actual help rather than just the appearance of safety.

The Paradox of Administrative Friction

The main failure of the Trusted Contact feature is the assumption that a user in crisis has the mental capacity to navigate complex settings. As NPR reported, the feature is not surfaced during standard interactions; users must manually find and configure it within the app settings.

Reading and understanding the instructions for how the feature works while someone is actively in crisis is likely to feel difficult and so that could be another barrier to getting help.

-- Ritu Chatterjee, reporting on expert consensus

This creates a friction gap. The system asks the user to perform a low-urgency, high-effort administrative task to solve a high-urgency, low-capacity problem. When the user is at their most vulnerable, the system asks them to act like a project manager. If they have not set up the tool during a period of stability, the feature is effectively useless.

The Illusion of Proactive Safety

OpenAI describes the feature as a safeguard, yet the system response to crisis triggers remains inconsistent. When a user asks the bot about the feature during distress, the bot may fail to provide guidance or, as noted in the transcript, may not recognize the intent.

The system relies on proactive offers that are still being tweaked. This creates a reliance on the bot correctly identifying distress and guiding the user toward a solution the user has not already set up. As psychologist Jennifer Lachman notes, the internal barriers for someone feeling suicidal are significant, specifically the feeling of being a burden. If the tool requires a technical workaround, the user is likely to give up rather than put in the effort to use the feature.

Systemic Limitations vs. Human Connection

Systems thinking requires us to look at what technology cannot replace. The Trusted Contact feature attempts to map a digital interaction to a human support network, but it lacks the nuance of a trained professional.

It does not and cannot simulate or replicate the process that you would go through with a mental health provider that is trained in identifying who those contacts should be.

-- Jennifer Lachman, Psychologist

The system treats human connection as a data point, such as a name and number, rather than a dynamic, emotional process. By offloading the identification of a trusted contact to the user, the system assumes the user has the clarity to know who is best suited to support them. When the system fails to bridge this gap, it shifts the burden back to the user, potentially deepening their sense of isolation if the solution provided by the AI is a dead end.

Key Action Items

  • Audit Safety UX for Cognitive Load: Evaluate current safety features to see if they require high executive function. If they do, they are likely failing when they are needed most. (Immediate)
  • Prioritize Proactive Onboarding: Move critical safety configurations into the initial user setup flow rather than burying them in settings menus. This ensures the infrastructure is in place before a crisis occurs. (Next 30 days)
  • Design for Crisis-State Interaction: Assume the user has zero capacity for complex instructions. Simplify all crisis-related UI to single-click or voice-activated paths. (Next 3-6 months)
  • Implement Feedback Loops with Domain Experts: Move beyond internal tweaking and establish formal, recurring audits with mental health professionals to test how the system behaves under simulated crisis conditions. (Ongoing)
  • Transparency on Efficacy: Acknowledge that good adoption metrics are not the same as effective intervention metrics. Invest in research that measures the actual impact on user outcomes rather than just feature usage. (12-18 months)

---
Handpicked links, AI-assisted summaries. Human judgment, machine efficiency.
This content is a personally curated review and synopsis derived from the original podcast episode.