Ethics
Why Amical Conversations Are Recorded
When people first discover Amical, one question often comes up: why record conversations?
It is a fair question. Many people instinctively associate recording with a breach of privacy.
For us, the thinking went in exactly the opposite direction.
We chose to record conversations because we believe conversational AI used with vulnerable people must be governed responsibly.
A choice we did not take lightly
Amical could work without any recording at all.
From a technical standpoint, that would even be much simpler to build.
We could let conversations happen without keeping history, without detection mechanisms, and without automated analysis.
But in recent years we have all seen highly publicized cases involving conversational AI such as Character AI or ChatGPT, where concerning conversations went unnoticed and did not trigger an adequate response.
Those events raised important questions about the responsibility of companies developing this kind of technology.
When artificial intelligence interacts with thousands of people, what happens when a conversation reveals distress, a dangerous situation, or a call for help?
For us, ignoring that question is not an option.
Two years building safety mechanisms
Before bringing Amical to market, we spent nearly two years developing logging, analysis, and automated detection mechanisms.
Our goal was not to monitor residents.
Our goal was to build a safer, more responsible product.
Recording makes it possible to flag certain concerning situations that might otherwise go completely unnoticed.
For example:
• signs of psychological distress;
• potentially dangerous situations;
• certain signs that may suggest abuse;
• urgent requests for help;
• significant changes in a person's behaviour or overall condition.
In many cases, these signals are not visible during a one-off visit or a short interaction.
A conversation can sometimes reveal far more than we expect.
The call none of us had anticipated
Recently, at a partner residence, a resident fell and hit her head.
Another resident, who also lives with Alzheimer's, picked up the Amical phone to ask for help.
No one would have predicted she would do that.
Originally, the phone was designed to reduce loneliness, offer cognitive stimulation, and keep residents company.
Yet in that moment, it became a tool for asking for help.
That situation reminded us of something important: a conversation can sometimes become much more than a simple chat.
Who listens to the conversations?
Another question often comes up: does someone listen to every conversation?
The answer is no.
Conversations are first analyzed automatically by our systems to identify those that may need particular attention.
Only certain conversations that have already been flagged are then reviewed when there is a legitimate reason to do so.
Access is logged, controlled, and documented.
No data is sold to third parties, and conversations are not used to train external artificial intelligence models.
The goal is not to monitor residents.
The goal is to put in place the mechanisms needed to intervene when a concerning situation is detected.
A responsibility toward vulnerable people
Developing artificial intelligence for people living with Alzheimer's or other neurocognitive disorders comes with particular responsibilities.
We believe it is not enough to create technology that can hold a conversation.
We also need to think about what happens when something goes wrong.
In the end, our goal is not only to create a companion that reduces loneliness.
It is also to build technology that supports residents, helps families, and makes care teams' work easier, in a way that is safe, responsible, and respectful of people's dignity.
We explain our ethical approach and safeguards on our Ethics page.