Usability and UX Testing in Chatbots

November 11, 2025

The growing use of AI-based chatbots has increased the need to understand how users perceive their usability and user experience (UX). This dissertation explores the challenges of evaluating chatbot us- ability and UX while examining the role of emotions as a complement to traditional assessment methods. Two AI-based chatbots, ChatGPT and DeepSeek, were compared through an experiment involving 36 participants who completed six tasks related to anxiety management and study strategies. The eval- uation combined usability and UX questionnaires (SUS, UEQ, CUQ, and BUS-11) with emotional as- sessment using the Self-Assessment Manikin (SAM). Results showed that ChatGPT achieved higher usability and UX scores, providing more coherent and engaging interactions. Strong correlations among the questionnaires supported their convergent validity, and SAM’s emotional dimensions, particularly Valence (satisfaction) and Dominance (sense of control), influenced perceived usability. Demographic factors such as prior chatbot experience and gender had minimal effects on evaluations. Overall, the findings indicate that combining emotional and traditional usability metrics offers a more comprehensive understanding of chatbot interactions. The study also confirms the effectiveness of CUQ and BUS- 11 in assessing chatbot usability, contributing to the development of more empathetic, adaptive, and user-centered conversational systems.

Authors in the community:

• 

  A

  Ana Aurora Xavier Nora

  ist193573

Supervisors of this institution:

• 

  S

  Sandra Pereira Gama

  ist152404

electrical-engineering-electronic-engineering-information-engineering - Electrical engineering, electronic engineering, information engineering

eng - English

Instituto Superior Técnico