Rational AIs with emotional deficits: ChatGPT vs. counselors in providing emotional reflections

Abstract

This study evaluates the effectiveness of human psychological counselors compared to ChatGPT in providing emotional reflection responses in therapeutic sessions. Employing a mixed-methods approach, we analyzed emotional reflection responses from both human counselors and ChatGPT to various client statements. In this study, quantitative data were evaluated using a 10-point Likert scale to measure the appropriateness of emotional reflections provided by both human counselors and ChatGPT in response to client statements, while qualitative insights for their reasons of choices were derived through thematic analysis. Quantitative results from the Mann-Whitney U-test reveal significant disparities in emotional reflection responses, with counselors outperforming ChatGPT in scenarios 2, 3, 4, 5, and 8 (medium to large effect sizes, r = 0.35 to 0.48), while ChatGPT excelled in scenarios 6 and 9, and no significant differences were found in scenarios 1, 7 (negligible effect size), and 10, highlighting AI limitations. Based on the qualitative findings of the study, we attribute these disparities to factors like therapeutic conditions, complexity of human emotions, differences between human and artificial intelligence regarding social and emotional intelligence, and the importance of professional training. We suggest that ChatGPT cannot replace the unique benefits of human counselors (at least for now), as it lacks the emotional depth and personal justification of a human touch. It could be used to support, not substitute, for emotional reflection responses, considering the potential drawbacks of relying solely on AI.