Human behavior can be seen as a window into their psychological states and mental conditions. One of the behavioral aspects of every individual that, to some extent, reflects their mental and cognitive states is their speech. Specifically, human speech and the way it is expressed—encompassing vocal and auditory features, tone, intensity, and speed—alongside the words employed to convey the speaker's message, serve as a platform for expressing emotions. From this perspective, speech signal processing and data mining can hold significance in the field of investigating mental health and computational psychology. Additionally, in the medical and clinical domain, one of the key factors in diagnosing and treating patients by attending physicians is obtaining a proper understanding of the patient's mental and emotional conditions. For instance, many functional disorders such as irritable bowel syndrome are heavily impacted by psychological factors including stress and anxiety. However, patients often restrain from sharing their emotions with physicians who lack expertise in psychology and psychiatry. On the other hand, treating physicians also lack the specialization or adequate time to uncover emotions and psychological contexts from the patient's speech. In this context, intelligent methods for emotion recognition from speech can serve as guiding and diagnostic aids. In this regard, this thesis addresses the issue of Speech Emotion Recognition (SER) using deep learning approaches. Notably, during the eva‎luation of methods and common structures, it was observed that the utilization of structures based on transformer models, previously applied in text processing, can perform well in processing speech signals and addressing the SER problem. To enhance accuracy and broaden the generality of the designed model, a combination of speeches in different languages was employed. This approach can be effective as a solution to tackle challenges associated with linguistic diversity and speech limitations. Moreover, in this research, apart from primary emotions, secondary emotions were also considered to construct a more practical model. Furthermore, in the direction of improving and further utilizing these methods in the patient-physician communication domain, a platform was developed for collecting patients' utterances and labeling them based on emotions, in the form of a web service.

جلال ذهبي

رسول امير فتاحي ورنوسفادراني , محمدرضا حيدرپور