THERMAL SCIENCE

International Scientific Journal

Authors of this Paper

External Links

Reem Alotaibi

ASPECT-BASED SENTIMENT ANALYSIS OF OPEN-ENDED RESPONSES IN STUDENT COURSE EVALUATION SURVEYS

ABSTRACT
Student surveys are tools used to gather feedback and opinions from students regarding various aspects of their educational experience such as satisfaction with courses, instructors, supervision, facilities, services, and overall campus environment. These surveys play a crucial role to assess and improve the quality of education and overall student experience, and provide insights that can be used to decision makers. There are two primary categories of survey questions: open-ended and closed-ended. Open-ended questions allow respondents to provide detailed answers in their own words, while closed-ended questions offer only predetermined options. For analyzing student surveys, it is common practice to focus on closed-ended questions, while often open-ended responses provide valuable insights. In this paper, we use sentiment analysis to extract students' emotions towards various aspects of their educational experience using transformer-based pre-trained language models. The results show that the multilingual XLM-RoBERTa demonstrated encouraging results compared to the multilingual BERT model with an accuracy of 0.81 and F1 of 0.80 for the classification of aspects while for the classification of sentiments, obtained 0.94 and 0.92 for accuracy and F1, respectively.
KEYWORDS
student surveys, topic modeling, sentiment analysis, BERT, course evaluation, open-ended responses, XLM-RoBERTa, aspect-based sentiment analysis
PAPER SUBMITTED: 2024-08-03
PAPER REVISED: 2024-10-20
PAPER ACCEPTED: 2024-11-21
PUBLISHED ONLINE: 2025-01-25
DOI REFERENCE: https://doi.org/10.2298/TSCI2406037A
CITATION EXPORT: view in browser or download as text file
THERMAL SCIENCE YEAR 2024, VOLUME 28, ISSUE Issue 6, PAGES [5037 - 5047]
REFERENCES
  1. Bell, A., Chris Brooks, C., What Makes Students Satisfied, A Discussion and Analysis of The Uk's National Student Survey, Journal of Further and Higher Education, 42 (2018), 8, pp. 1118-1142
  2. Mizrachi, D., et al., Academic Reading Format Preferences and Behaviors among University Students World-Wide: A Comparative Survey Analysis, PLOS ONE, 13 (2018), 05, pp. 1-32
  3. Marshall, P., Contribution of Open-Ended Questions in Student Evaluation of Teaching, Higher Education Research and Development, 41 (2022), 6, pp. 1992-2005
  4. Baburajan, V., et al., Open-Ended Versus Closed-Ended Responses: A Comparison Study Using Topic Modeling and Factor Analysis, IEEE Tran. on Inte. Trans. Systems, 22 (2021), 4, pp. 2123-2132
  5. Brauwers, G., Flavius Frasincar, F., A Survey on Aspect-Based Sentiment Classification, ACM Computing Survey, 55 (2021), 4
  6. Sun, Q., et al., Exploring Ewom in Online Customer Reviews: Sentiment Analysis at a Fine-Grained Level, Engineering Applications of Artificial Intelligence, 81 (2019), May, pp. 68-78
  7. Shaik, T., et al., Sentiment Analysis and Opinion Mining on Educational Data: A Survey, Natural Language Processing Journal, 2 (2023), 100003
  8. Zhang, W., et al., A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges, IEEE Transactions on Knowledge and Data Engineering, 35 (2023), 11, pp. 11019-11038
  9. Mowlaei, M. E., et al., Aspect-Based Sentiment Analysis Using Adaptive Aspect-Based Lexicons, Expert Systems with Applications, 148 (2022), 113234
  10. Casola, S., et al., Pre-Trained Transformers: An Empirical Comparison, Machine Learning with Applications, 9 (2022), 100334
  11. Liu, P., et al., Pretrain, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing, ACM Computing Surveys, 55 (2023), 9, pp. 1-35
  12. Sindhu, I., et al., Aspect-Based Opinion Mining on Student's Feedback for Faculty Teaching Performance Evaluation, IEEE Access, 7 (2019), July, pp. 108729-108741
  13. Kastrati, Z., et al., Weakly Supervised Framework for Aspect-Based Sentiment Analysis on Students' Reviews of Moocs, IEEE Access, 8 (2020), June, pp. 106799-106810
  14. Sultani M., Daneshpour, N., Extracting Urgent Questions from Mooc Discussions: A Bertbased Multi-Output Classification Approach, Arabian Journal for Science and Engineering, On-line first, doi.org/10.1007/s13369-024-09090-7, 2024
  15. Hussain, S., et al., Aspect2labels: A Novelistic Decision Support System for Higher Educational Institutions by Using Multi-Layer Topic Modelling Approach, Expert Systems with Applications, 209 (2022), 118119
  16. Ren, R., et al., Automatic Scoring of Student Feedback for Teaching Evaluation Based on Aspect-Level Sentiment Analysis, Education and Information Technologies, 28 (2022), July, pp. 797-814
  17. Rogers, A., et al., A primer in BERTology: What We Know about How BERT Works, Transactions of the Association for Computational Linguistics, 8 (2020), Dec., pp. 842-866
  18. Zhang, H., et al., A Bert Fine-Tuning Model for Targeted Sentiment Analysis of Chinese Online Course Reviews, International Journal on Artificial Intelligence Tools, 29 (2020), 07-08, 2040018
  19. Alshaikh, K. A., et al., et al., Bert-Based Model for Aspect-Based Sentiment Analysis for Analyzing Arabic open-Ended Survey Responses: A Case Study, IEEE Access, 12 (2024), Dec., pp. 2288-2302
  20. Mustafa, M., et al., Arabic Aspect Sentiment Polarity Classification Using Bert, Journal of Big Data, 9 (2022), 12
  21. Chennafi, M. E., et al., Arabic Aspect-Based Sentiment Classification Using Seq2Seq Dialect Normalization and Transformers, Knowledge, 2 (2022), 3, pp. 388-401
  22. AlJarrah, I., et al., Aspect-Based Sentiment Analysis for Arabic Food Delivery Reviews, ACM Transactions on Asian and Low-Resource Language Information Processing, 22 (2023), July, pp. 2375-4699
  23. Hasna Chouikhi, H., Mohammed Alsuhaibani, and Fethi Jarray. Bertbased Joint Model for Aspect Term Extraction and Aspect Polarity Detection in Arabic Text, Electronics, 12 (2023), 3, pp. 2079-9292
  24. Almasri, M., et al., A Semi Supervised Approach to Arabic Aspect Category Detection Using Bert and Teacher-Student Model, PeerJ Computer Science, 9 (2023), e1425
  25. Bensoltane R., Zaki, T., Towards Arabic Aspect-Based Sentiment Analysis: A Transfer Learning-Based Approach, Social Network Analysis and Mining, 12 (2021), Nov., pp. 1-16
  26. El-Alami, F. Z., et al., Contextual Semantic Embeddings Based on Fine-Tuned Arabert Model for Arabic Text Multi-Class Categorization, Journal of King Saud University-Computer and Information Sciences, 34 (2022), 10, pp. 8422-8428
  27. Li, B., et al., Robust Dimensionality Reduction Via Feature Space to Feature Space Distance Metric Learning, Neural Networks, 11 (2019), 2, pp. 1-14
  28. Campello, R. J. G. B., et al., Data Mining and Knowledge Discovery, 10 (2020), 2, e1343
  29. George, L., Sumathy, P., An Integrated Clustering and Bert Framework for Improved Topic Modeling International Journal of Information Technology, 15 (2023), 4, pp. 2187-2195
  30. Abuzayed, A., Al-Khalifa, H., Bert for Arabic Topic Modeling: An Experimental Study on Bertopic Technique, Procedia Computer Science, 189 (2021), 1, pp. 191-194
  31. Miao, J., Zhu, W., Precision-Recall Curve (PRC) Classification Trees, Evolutionary Intelligence, 15 (2022), 09
  32. Christen, P., et al., A Review of the F-Measure: Its History, Properties, Criticism, and Alternatives, ACM Computing Surveys, 56 (2023), 3, pp. 1-24
  33. Xu, J., et al., Three-Way Confusion Matrix for Classification: Ameasure Driven View, Information Sciences, 507 (2020), Jan., pp. 772-794
PDF VERSION [DOWNLOAD]
Aspect-based sentiment analysis of open-ended responses in student course evaluation surveys

2025 Society of Thermal Engineers of Serbia. Published by the Vinča Institute of Nuclear Sciences, National Institute of the Republic of Serbia, Belgrade, Serbia. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International licence