Introduction
Fifteen-year-old Jordyne Lewis was stressed out. Overwhelmed with schoolwork and pandemic anxieties, the North Carolina high school sophomore never turned to her school counsellor or sought out a therapist. Instead, she shared her feelings with a robot- Woebot, an AI chatbot.[1] When feeling “nervous and overwhelmed” about upcoming tests, Lewis reported feeling better after the chatbot urged her to challenge negative thoughts and offered breathing exercises. “It’s a robot,” she explained. “It’s objective. It can’t judge me.” Yet Lewis herself acknowledged a crucial limitation: “It’s like the stepping stone to getting help, but it’s definitely not a permanent solution.”[2] This distinction, between AI’s ability to simulate supportive responses and its capacity to genuinely understand human emotion raises a fundamental question: Can artificial intelligence, from simple voice assistants like Siri to sophisticated humanoid robots like Sophia, ever truly comprehend the depths of human empathy and emotional experience?
Emotional Intelligence in AI Systems
Modern AI systems demonstrate sophisticated capabilities in detecting, analysing and responding to human emotions through affective computing, a field that combines facial recognition, voice analysis and text sentiment processing.[3] These systems employ deep learning techniques and multimodal data fusion, enabling them to recognize emotional states by analysing multiple signals simultaneously: micro-expressions in faces, vocal tone variations and linguistic patterns in text.[4] The technology processes these inputs through neural networks trained on vast datasets of human emotional expressions, generating contextually appropriate responses in real-time.
Empathetic algorithms have found practical applications across healthcare, education and customer service sectors.[5] Mental health platforms like Woebot and Wysa provide 24/7 emotional support, offering cognitive behavioural therapy techniques and crisis intervention when human therapists are unavailable. In educational settings, AI systems personalize encouragement and adapt instructional approaches based on student emotional states, while customer service applications adjust communication tone to de-escalate frustrated users.[6]
Research involving 985 participants comparing empathy toward AI-generated versus human-written emotional narratives reveals that users often report feeling “heard” and “understood” by emotional AI systems.[7] Reports reveal emotion detection AI models can achieve accuracy rates of approximately 96% in recognizing up to seven different emotions from facial expressions in controlled settings[8], while convolutional neural network models demonstrate test accuracy of 95% with high precision across all emotion categories.[9] Just how curiously concerning are such statistics?
The Limits of Simulated Empathy
While AI demonstrates impressive emotional recognition capabilities, research reveals important distinctions between simulating empathy and genuinely experiencing it. Sophia, the humanoid robot created by Hanson Robotics, has sparked considerable debate within the AI community. Though marketed with claims of consciousness and emotional capacity, experts largely categorize Sophia as “a chatbot with a face,”[10] highlighting the gap between public perception and technical reality. This case illustrates broader questions about what constitutes authentic emotional understanding.
A Cornell University study presented at the CHI ‘24 conference tested conversational agents’ displays of empathy across 65 distinct human identities, revealing that large language models make value judgments and exhibit biases rather than authentic empathetic responses.[11] This represents what researchers term the “illusion of empathy”- pattern recognition algorithms matching emotional cues to pre-programmed responses without subjective experience. Philosopher Thomas Fuchs’s phenomenological analysis demonstrates that genuine empathy requires preconditions fundamentally absent in artificial systems: embodied experience, intersubjective consciousness and the capacity for emotional resonance.[12] Recent research on digitalized therapy confirms this unresolved gap, showing that AI lacks the lived experience necessary to genuinely care about therapeutic outcomes.[13] Understanding this distinction proves critical for ethical deployment of emotional AI in contexts demanding authentic human connection.
Simulated Empathy – Risks, Human Necessity and Ethical Frontiers
These AI systems such as Woebot, Replika and ElliQ are increasingly portrayed as companions capable of empathy. Yet, their empathy is not genuine but simulated an algorithmic imitation of human care. Unlike authentic empathy, which arises through lived experience and shared emotional understanding, simulated empathy performs understanding without truly feeling it. A 2024 study published in Frontiers in Psychology found that many users attributed emotional and moral awareness to AI companions even after learning that these systems lacked consciousness.[14] This response highlights a deeper human longing to be understood rather than any real empathic ability on the part of machines.
Such emotional reliance, however, carries notable risks. Research from Stanford University’s Social AI Lab (2024) warns that constant interaction with emotionally responsive chatbots may diminish individuals’ coping mechanisms and discourage help-seeking behaviour, especially among vulnerable groups like adolescents and the elderly.[15] When users begin to treat AI as emotionally trustworthy, the line between comfort and dependency blurs.
The ethical dimension of this issue is equally pressing. If an AI system provides advice or comfort that leads to harm, who is morally or legally accountable? Scholars have termed this “emotional deception” – the appearance of care without any genuine feeling or responsibility.[16] Philosopher Thomas Fuchs argues that real empathy involves “existential reciprocity,” a mutual awareness of being understood by another, which no algorithm can replicate.[17]
Moreover, emotional AI often reflects cultural bias. A report found that emotion-recognition systems trained on Western facial data misread non-Western expressions by up to 25%.[18] To counter these challenges, advocates for “empathy disclaimers” and hybrid human-AI models to ensure that technology enhances, rather than replaces authentic human connection.[19]
Conclusion
While machines from Siri to Sophia can convincingly simulate emotional responses, they lack the subjective, embodied, inter-relational basis of real empathy. While these systems can offer comfort, provide mental health support or serve as companions in controlled settings, they cannot replace the emotional depth, moral understanding or human connection that comes from lived experience.
The risks are significant: over-reliance on AI for emotional support can foster dependency, weaken coping skills and blur the line between real and simulated care. Biases in emotion-recognition systems and the possibility of emotional deception further highlight the need for transparency and ethical oversight. Ultimately, the question is not whether AI can truly feel, but whether we, in our longing to be understood, might mistake its simulation for real empathy. Human empathy remains irreplaceable and it is our responsibility to preserve it in a rapidly digitizing world.
[1] Mark Keierleber, Young and Depressed? Try Woebot! The Rise of Mental Health Chatbots in the US, The Guardian (Apr. 13, 2022), https://www.theguardian.com/us-news/2022/apr/13/chatbots-robot-therapists-youth-mental-health-crisis accessed 29 October 2025.
[2] Id.
[3] J. Narimisaei et al., Exploring Emotional Intelligence in Artificial Intelligence Systems: A Comprehensive Analysis of Emotion Recognition and Response Mechanisms, (2024), https://pmc.ncbi.nlm.nih.gov/articles/PMC11305735/ accessed 28 September 2025.
[4] Id.
[5] S. B. Velagaleti et al., Empathetic Algorithms: The Role of AI in Understanding and Enhancing Human Emotional Intelligence, 20 J. Elec. Sys. 2051, 2051–60 (2024), https://journal.esrgroups.org/jes/article/view/1806 accessed on 28 September 2025.
[6] Id.
[7] Jingwen Shen et al., Empathy Toward Artificial Intelligence Versus Human Experiences and the Role of Transparency in Mental Health and Social Support Chatbot Design: Comparative Study, 11 JMIR Mental Health, e62679 (2024), https://mental.jmir.org/2024/1/e62679 accessed on 28 September 2025.
[8] Narimisaei J and others, supra note 1, at 3.
[9] H. Vicci, Emotional Intelligence in Artificial Intelligence: A Review and Evaluation Study, SSRN Electronic Journal (May 2024), https://www.researchgate.net/publication/380457655_Emotional_Intelligence_in_Artificial_Intelligence_A_review_and_evaluation_study accessed 29 September 2025.
[10] Catherine Ewell, Bringing AI to Life, WIPO Magazine (Sept. 2018), https://www.wipo.int/wipo_magazine/en/2018/05/article_0002.html accessed 29 September 2025.
[11]Tom Fleischman, AI-Generated Empathy Has Its Limits, Cornell Chronicle (May 8, 2024), https://news.cornell.edu/stories/2024/05/ai-generated-empathy-has-its-limits accessed 29 September 2025.
[12] Thomas Fuchs, Understanding Sophia? On Human Interaction with Artificial Agents, 23 Phenomenology & Cognitive Sci. 21, 21–42 (2024), https://link.springer.com/article/10.1007/s11097-022-09848-0 accessed 29 September 2025.
[13] R. Salil et al., Digitalized Therapy and the Unresolved Gap Between Artificial and Human Empathy, 15 Frontiers Psychiatry, 1522915 (2025), https://pmc.ncbi.nlm.nih.gov/articles/PMC11752889/ accessed 29 September 2025.
[14] Frontiers in Psychology, Attribution of Empathy to Artificial Companions: User Perception and Moral Projection (2024), https://www.frontiersin.org/journals/psychology accessed 13 November 2025.
[15]Stanford Univ. Social AI Lab, Emotional Dependence in Human–AI Interaction: Risks for Vulnerable Populations (2024).
[16] Rilbert Teixeira, Emotional AI Applied to Mental Health: An Ethical and Philosophical Analysis, in Lecture Notes in Computer Science, vol. 16108 (2025), https://link.springer.com/chapter/10.1007/978-3-032-04999-5_25 accessed 10 November 2025.
[17] Thomas Fuchs, Empathy, Intersubjectivity, and the Limits of Artificial Simulation, 21 Phenomenology & Cognitive Sci. 3 (2022).
[18] Yen-Ting Li, Su-Ling Yeh & Tsung-Ren Huang, The Cross-Race Effect in Automatic Facial Expression Recognition Violates Measurement Invariance, 14 Frontiers Psychol. 1201145 (2023), https://doi.org/10.3389/fpsyg.2023.1201145 accessed 12 November 2025.
[19] Jingwen Shen et al., Empathy Toward Artificial Intelligence Versus Human Experiences and the Role of Transparency in Mental Health and Social Support Chatbot Design: Comparative Study, 11 JMIR Mental Health, e62679 (2024), <doi:10.2196/62679> accessed 13 November 2025.
