سامانه همایش های دانشگاه آزاد اسلامی واحد نجف آباد

صفحه اصلی / سومین کنفرانس بین المللی مهندسی و علوم کامپیوتر و اولین کنفرانس ملی سامانه های هوشمند، کلان داده و رایانش پیشرفته

A Comparative Study of Transformer–GRU Models with Advanced Loss Optimization for Sentiment Analysis across Short and Long Texts

نویسندگان :

Bahar Asgari¹ Hamid Rastegari² Vahid Nejati³

1- دانشگاه آزاد اسلامی واحد نجف آباد 2- دانشگاه ازاد اسلامی واحد نجف اباد 3- دانشگاه شهید بهشتی

کلمات کلیدی :

Sentiment Analysis،Deep Learning،BERT،DistilBERT،GRU،Loss Function

چکیده :

Sentiment analysis provides a crucial mechanism for information extraction from textual data; however, its effectiveness is influenced by model architecture, dataset characteristics, and training strategies. This work presents a systematic evaluation of hybrid sentiment classification frameworks that combine transformer-based encoders like BERT and DistilBERT with Gated Recurrent Units (GRUs), focusing on the role of loss function design and training dynamics rather than model size alone. Several optimization strategies—including dynamic loss scheduling, dynamic categorical cross-entropy, confidence-aware loss, and confidence-weighted categorical cross-entropy—are examined in conjunction with cosine learning rate decay. Experiments are conducted on two benchmark datasets with contrasting properties: IMDb, representing long and relatively structured reviews, and Sentiment140, consisting of short and informal social media texts. Experimental results show that DistilBERT+GRU provides stable and competitive performance across both datasets, achieving up to 98.37% accuracy on IMDb and 97.29% on Sentiment140, while maintaining balanced precision, recall, and F1 scores. In contrast, BERT+GRU exhibits greater sensitivity to optimization settings, particularly on long-text data. Although RoBERTa+GRU attains comparable accuracy on short texts, its overall gains over DistilBERT-based models remain limited. The findings highlight that effective loss design and training dynamics can significantly enhance model stability and narrow the performance gap between lightweight and larger transformer architectures.