What you type in ChatGPT can now, officially, end up on an investigator’s desk. In a post titled Our commitment to community safetypublished at the end of April, OpenAI details how its systems monitor conversations to detect risks of violence.
The context is heavy: the post opens with mass shootings and threats against elected officials. The media are finally recalling a recent killing in Canada, preceded by disturbing messages sent to ChatGPT. After judging the danger to be insufficient at the time, Sam Altman today formalizes a much more intrusive device.
ChatGPT and OpenAI: what does this post really say which formalizes surveillance?
The post first describes an automatic detection chain: classifiers continuously scan messages to identify threats, intentions to purchase weapons or fantasies of an attack. OpenAI says it clearly: “An isolated message may seem harmless, but a pattern that emerges over a long conversation or between several sessions can reveal something more serious.”
When a conversation is “flagged,” employees and mental health experts reread the entire exchange, including the history over several weeks. Objective: establish a behavioral profile, assess whether the threat is credible and decide on a possible escalation.
Last stage of the rocket, OpenAI reserves the right to notify law enforcement in the event of danger deemed “imminent and credible”. Without a judge, without a warrant and without an independent audit, the company explains that “Our reporting criteria remain flexible, because a user does not always explicitly mention the target, means or timetable of planned violence.”
ChatGPT: what this monitoring actually changes for your uses
This channel is not only used to filter content to train models, but to analyze your behavior over time. Even if you disable the use of your chats for learning, the security layer remains active and continues to produce risk scores.
For minors, OpenAI goes further: human examiners read certain teenagers’ conversations and can trigger an alert to parents in the event of “acute distress” detected. A family conflict, a complicated coming out or dark thoughts confided to ChatGPT then risk returning home, sometimes at the worst moment.
The post also announces a “trusted contact” option for adults: you can designate a person that the AI will notify if it considers that you need support. Assumed paternalism, no statistics published on the number of conversations read or the number of reports made.
OpenAI, authorities and legal framework: what can users do now?
On the transparency side, the text remains vague: no external audit, no figures, no explicit mention of GDPR or the European legal framework. Only an internal appeal mechanism is provided, which amounts to letting the supervisor judge his own supervision.
For a French user, the line to follow is therefore pragmatic: consider that any potentially incriminating or overly intimate message can, one day, be reread by a human or transmitted to the authorities. Avoid detailed medical confidences, full confessions or threats, even “in jest”: ChatGPT is neither a psychologist, nor a lawyer, nor a diary protected by secrecy.