A Hybrid Framework for Persian Sentiment Analysis, Integrating Transformer-based Language Models, Aspect-based Analysis, and Confidence Calibration

Ziafat, Hasan; Dorostkat, Behnam; Rahmani, Parisa; Fareghzadeh, Nafiseh

A Hybrid Framework for Persian Sentiment Analysis, Integrating Transformer-based Language Models, Aspect-based Analysis, and Confidence Calibration

Document Type : Original Research Article

Authors

Hasan Ziafat ¹

Behnam Dorostkat ² ^¶

Parisa Rahmani ³

Nafiseh Fareghzadeh ⁴

¹ Department of computer engineering,Nat.C., Islamic Azad University,Natanz, Iran

² Department of Information and Communication, Amin Police University, Tehran, Iran

³ Department of Computer Engineering, Islamic Azad University, Par.C., Tehran, Iran

⁴ 3 Department of Computer Science, Islamic Azad University, Khodabandeh Branch, Zanjan, Iran

Abstract

Sentiment analysis is a key application of natural language processing and artificial intelligence, aimed at identifying and extracting attitudes, emotions, and polarity from human-generated text. Although deep learning models—particularly Transformer-based architectures—have significantly enhanced the capabilities of sentiment analysis systems, analyzing Persian text remains fraught with challenges such as writing variations, the use of zero-width non-joiners (ZWNJ), character form variations, semantic ambiguity, sarcasm, negation, multi-aspect perspectives, a scarcity of labeled data, and a mismatch between model confidence and actual prediction accuracy. Furthermore, many existing systems merely output a general positive, negative, or neutral label without accounting for their confidence in the decision. To address these challenges, this research proposes a multi-stage, context-aware, and uncertainty-driven framework for Persian sentiment analysis. This framework comprises four main layers: Persian preprocessing and normalization, deep representation extraction using Transformer models, aspect-based sentiment analysis, and finally, calibration and uncertainty estimation. This structure enables the distinction between "overall sentiment" and "sentiment regarding a specific aspect." Evaluation results demonstrate that the proposed framework outperforms baseline models across all metrics, including Accuracy, Precision, Recall, and Macro-F1. Additionally, an SCCI value of 91% indicates a high level of consistency and coherence in the model's ability to identify aspect-level sentiment within Persian texts. These results demonstrate that employing the proposed architecture has led to significant improvements not only in classification performance but also in the balanced and simultaneous detection of emotion labels.

Keywords

Natural Language Processing

Transformational

Aspect-Based Sentiment Analysis

Calibration

Deep Learning

[1] B. Liu, Sentiment Analysis and Opinion Mining. Cham, Switzerland: Springer, 2012, doi: 10.1007/978-3-031-02145-9.

[2] B. Pang and L. Lee, “Opinion Mining and Sentiment Analysis,” Foundations and Trends in Information Retrieval, vol. 2, no. 1–2, pp. 1–135, 2008, doi: 10.1561/1500000011.

[3] M. Pontiki, D. Galanis, J. Pavlopoulos, H. Papageorgiou, I. Androutsopoulos, and S. Manandhar, “SemEval-2014 Task 4: Aspect Based Sentiment Analysis,” in Proc. 8th Int. Workshop on Semantic Evaluation, 2014, pp. 27–35, doi: 10.3115/v1/S14-2004.

[4] M. Pontiki et al., “SemEval-2016 Task 5: Aspect Based Sentiment Analysis,” in Proc. 10th Int. Workshop on Semantic Evaluation, 2016, pp. 19–30, doi: 10.18653/v1/S16-1002.

[5] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proc. NAACL-HLT, 2019, pp. 4171–4186, doi: 10.18653/v1/N19-1423.

[6] Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” arXiv:1907.11692, 2019.

[7] A. Vaswani et al., “Attention Is All You Need,” in Advances in Neural Information Processing Systems, 2017.

[8] A. Conneau et al., “Unsupervised Cross-lingual Representation Learning at Scale,” in Proc. ACL, 2020.

[9] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, “On Calibration of Modern Neural Networks,” in Proc. ICML, vol. 70, pp. 1321–1330, 2017.

[10] A. Esuli and F. Sebastiani, “SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining,” in Proc. LREC, 2006.

[11] A. Esuli and F. Sebastiani, “Determining the Semantic Orientation of Terms through Gloss Classification,” in Proc. CIKM, 2005.

[12] A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts, “Learning Word Vectors for Sentiment Analysis,” in Proc. ACL-HLT, 2011, pp. 142–150.

[13] S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997.

[14] K. Cho et al., “Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation,” in Proc. EMNLP, 2014.

[15] N. Reimers and I. Gurevych, “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,” in Proc. EMNLP-IJCNLP, 2019, pp. 3982–3992, doi: 10.18653/v1/D19-1410.

[16] M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer, “Deep Contextualized Word Representations,” in Proc. NAACL-HLT, 2018, pp. 2227–2237, doi: 10.18653/v1/N18-1202.

[17] L. Ouyang et al., “Training Language Models to Follow Instructions with Human Feedback,” arXiv:2203.02155, 2022.

[18] J. Howard and S. Ruder, “Universal Language Model Fine-tuning for Text Classification,” arXiv:1801.06146, 2018.

[19] T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient Estimation of Word Representations in Vector Space,” arXiv:1301.3781, 2013.

[20] J. Pennington, R. Socher, and C. Manning, “GloVe: Global Vectors for Word Representation,” in Proc. EMNLP, 2014.

[21] Y. Kim, “Convolutional Neural Networks for Sentence Classification,” in Proc. EMNLP, 2014.

[22] R. Socher et al., “Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank,” in Proc. EMNLP, 2013.

[23] A. Joulin, E. Grave, P. Bojanowski, and T. Mikolov, “Bag of Tricks for Efficient Text Classification,” in Proc. EACL, 2017.

[24] T. Wolf et al., “Transformers: State-of-the-Art Natural Language Processing,” in Proc. EMNLP: System Demonstrations, 2020.

[25] T. Brown et al., “Language Models are Few-Shot Learners,” in Advances in Neural Information Processing Systems, vol. 33, 2020.

[26] W. Fedus, B. Zoph, and N. Shazeer, “Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity,” Journal of Machine Learning Research, vol. 23, 2022.

[27] T. Kudo and J. Richardson, “SentencePiece: A Simple and Language Independent Subword Tokenizer and Detokenizer for Neural Text Processing,” in Proc. EMNLP: System Demonstrations, 2018.

[28] M. Farahani, M. Gharachorloo, M. Farahani, and M. Manthouri, “ParsBERT: Transformer-based Model for Persian Language Understanding,” Neural Processing Letters, 2021, doi: 10.1007/s11063-021-10528-4.

[29] P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, “Enriching Word Vectors with Subword Information,” Transactions of the Association for Computational Linguistics, vol. 5, pp. 135–146, 2017.

[30] N. Fareghzadeh, M. A. Seyyedi, and M. Mohsenzadeh, “Dynamic Performance Isolation Management for Cloud Computing Services,” The Journal of Supercomputing, vol. 74, no. 1, pp. 417–455, 2018, doi: 10.1007/s11227-017-2135-2.

[31] N. Fareghzadeh, M. A. Seyyedi, and M. Mohsenzadeh, “Toward Holistic Performance Management in Clouds: Taxonomy, Challenges and Opportunities,” The Journal of Supercomputing, vol. 75, no. 1, pp. 272–313, 2019, doi: 10.1007/s11227-018-2679-9.

[32] N. Fareghzadeh, “An Architecture Supervisor Scheme toward Performance Differentiation and Optimization in Cloud Systems,” The Journal of Supercomputing, vol. 78, no. 8, pp. 1532–1563, 2022, doi: 10.1007/s11227-021-03846-w.

[33] N. Fareghzadeh, “Service Identification Approach to SOA Development,” International Journal of Computer and Information Engineering, vol. 2, no. 9, pp. 2970–2978, 2008.

[34] M. Bazargani, N. Fareghzadeh, M. Afzali, and S. Karimi, “A Hybrid Cumulative Knowledge Framework for Friend Recommendation in Social Networks,” International Journal of Nonlinear Analysis and Applications, 2025, doi: 10.22075/ijnaa.2024.34433.5144.

[35] H. Jafarian, A. H. Taghavi, A. Javaheri, and R. Rawassizadeh, “Exploiting BERT to Improve Aspect-Based Sentiment Analysis Performance on Persian Language,” in Proc. 7th Int. Conf. Web Research (ICWR), 2021, pp. 5–8, doi: 10.1109/ICWR51868.2021.9443131.

[36] B. Pang, L. Lee, and S. Vaithyanathan, “Thumbs Up? Sentiment Classification using Machine Learning Techniques,” in Proc. EMNLP, 2002.

[37] A. Pak and P. Paroubek, “Twitter as a Corpus for Sentiment Analysis and Opinion Mining,” in Proc. LREC, 2010.

[38] E. Cambria, B. Schuller, Y. Xia, and C. Havasi, “New Avenues in Opinion Mining and Sentiment Analysis,” IEEE Intelligent Systems, vol. 28, no. 2, pp. 15–21, 2013.

[39] E. Cambria, D. Das, A. Feraco, A. Hussain, and C. Havasi, A Practical Guide to Sentiment Analysis. Springer, 2017.

[40] A. B. Nassif, A. M. Darya, and A. Elnagar, “Empirical Evaluation of Shallow and Deep Learning Classifiers for Arabic Sentiment Analysis,” ACM Trans. Asian Low-Resour. Lang. Inf. Process., vol. 21, no. 1, Art. no. 14, pp. 1–25, 2022, doi: 10.1145/3466171.

[41] T. Shangipour Ataei, K. Darvishi, S. Javdan, B. Minaei-Bidgoli, and S. Eetemadi, “Pars-ABSA: A Manually Annotated Aspect-Based Sentiment Analysis Benchmark on Farsi Product Reviews,” in Proc. 13th Language Resources and Evaluation Conference (LREC), Marseille, France, 2022, pp. 7056–7060.