Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.
Advertisement
Scientific Reports volume 14, Article number: 31928 (2024)
Metrics details
The increasing development of technology has led to the increase of digital data in various fields, such as medication-related texts. Sentiment Analysis (SA) in medication is essential to give clinicians insights into patients’ feedback about the treatment procedure. Therefore, this study intends to develop Artificial Intelligence (AI) models to predict patients’ sentiments. This study used a large medication review dataset to perform a SA of medications. Three scenarios were considered for classification, including two, three, and ten classes. The Word2Vec algorithm and pre-trained word embeddings, including the general and clinical domains, were utilized in model development. Seven Machine Learning (ML) and Deep Learning (DL) models were developed for various scenarios. The best hyperparameters for all models were fine-tuned. Moreover, two ensemble learning models were developed from the proposed ML and DL models. For the first time, a technique was implemented to interpret the results for explainability and interpretability. The results showed that the developed deep ensemble model (DL_ENS), using PubMed and PMC, as pre-trained word embedding representation, achieved the best results, with accuracy and F1-Score of 92.96% and 92.27% in two classes, 92.18% and 88.50 in three classes, and 90.31% and 67.07% in ten classes, respectively. Combining DL models and developing a DL_ENS with clinical domain pre-trained word embedding representation can accurately predict classes and scores of patients’ sentiments about medications compared to previous studies on the same dataset. Due to the transparency in decision-making, our DL_ENS model can be used as an auxiliary tool to help clinicians prescribe medications.
In recent decades, users’ reviews on websites and social media have been increased dramatically. Users publish their reviews about various products such as medication, movies, restaurants, home appliances, clinical services, and marketing on related sites. People read the reviews before purchasing or using products or services, which enables them to make informed decisions based on previous reviews1,2,3.
Medical and clinical reports, patients’ feedback, and sentiments about medical systems and services are among the most valuable and useful textual content4,5,6. Although the safety of medications after production is monitored and tested under standard clinical conditions, people still write their reviews and sentiments on medication review websites. These websites enable people to read reviews about the medications before using them. Clinicians and pharmacists can also use this information to align with patient expectations and experiences, ultimately improving adherence and treatment effectiveness. Sentiment analysis of patient feedback enables clinicians to adjust prescribing behaviors by recognizing common patient-reported side effects and satisfaction levels associated with medications. This approach enhances clinicians’ decision-making process and informs patient care strategies by highlighting areas where medication efficacy or patient experience could be improved4,5,6.
Analyzing such text data using sentiment analysis (SA) is a useful step in evaluating medications’ effectiveness and side effects more accurately4,5,7,8,9,10,11. Prediction of patients’ sentiments in medicine can be used to facilitate treatment in the future because the results of different types of medical treatments and awareness of their effectiveness have been investigated. Analyzing patients’ sentiments in clinical documents, where treatment processes and results were recorded from the past to the present, can help clinicians and pharmacists identify factors affecting diseases and treatments12.
SA is one of the most prominent branches of natural language processing that focuses on finding the sentiments in the texts13. In SA, reviews are divided into different levels, including aspect, entity, sentence, and document levels13. SA is known as a classification problem, and some studies used artificial intelligence (AI) techniques to solve this problem6. Machine Learning (ML) and Deep Learning (DL) algorithms are branches of AI, and their use in clinical fields, especially in the field of clinical SA and text mining, has recently increased6,8.
Many studies in the past have used AI techniques to detect sentiments in non-clinical contexts1,2,3,14,15,16,17,18,19,20. However, previous studies in the clinical and medicine domain6,8,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39, which are reviewed in Sect. 2, have various shortcomings:
Various scenarios for predicting sentiment have not been considered.
Most studies have not considered finding the best hyperparameters for their proposed models.
Most studies in this field utilized relatively small datasets.
The impact of using various pre-trained word embeddings in the clinical domain has not been studied.
The explainability and interpretability of AI models, which are very important in the clinical domain, have not been considered in previous studies.
To address these shortcomings, this study aims to provide a new method for comprehensively identifying the sentiments of medication reviews. For this aim, seven AI techniques, including ML and DL models and two ensemble (ENS) models, were developed to predict patients’ sentiments and rate scores.
In general, our main contributions in this study are as follows:
Proposed ML and DL models were trained and tested on the drugs.com dataset, which is a rich dataset for medication reviews.
Three scenarios have been considered to comprehensively determine patients’ sentiment classes and rate scores.
Various ML, DL, and ENS algorithms and experiments are developed to compare the prediction of patients’ sentiments and rate scores.
A deep ensemble (DL_ENS) model with a new architecture is designed to predict patients’ sentiments and rate scores in medication reviews.
The best values of the hyperparameters of ML and DL models have been found and selected for prediction using Grid Search.
Various general and clinical pre-trained word embedding models were used to improve AI models’ performance.
This study is the first study using Local Interpretable Model-agnostic Explanations (LIME) to explain the decision-making process to have interpretable and explainable AI models for medication reviews.
The remainder of this paper is divided into six sections: Section two presents an overview of previous studies on SA, Section three describes the materials and methods that have been used in this study, Section four presents the results, Section five is the discussion, and finally, Section six concludes the study and outlines future work.
Table 1 provides an overview of previous SA studies including their methods, type of dataset, and their research gaps6,8,15,16,17,18,19,21,24,25,28,29,30,31,32,35,37,39.
Various researchers have attempted to analyze the medication reviews provided by people to get insight into their conditions, needs, sentiments, and the side effects of each medication6,8,28,29,30,31,32,33,34,35,36,37,38,39. SA can play an important role in this field for extracting beneficial information for mentioned purposes. Most studies in this field are based on datasets with a limited number of samples. Moreover, most researchers have used rule-based and traditional ML models for SA31.
For SA in medication reviews, datasets with a more significant number of samples by ML algorithms such as Support Vector Machine (SVM), Logistic Regression (LR), Naïve Bayes (NB), Decision Tree (DT), Random Forest (RF), K Nearest Neighbor (KNN) were considered28,30,32,38,39. Recently, ML methods were also used for SA in medication reviews in a non-English language, and their performance was compared with medication reviews in English32. Moreover, new feature extraction methods and ML algorithms have been proposed to improve the performance of models in SA of medication reviews39.
In the SA of medication reviews, DL algorithms, including Convolutional Neural Networks (CNN), have been developed to achieve better performance in sentiment classification8,35,36. With the increasing use of DL models, the performance of these models has also been compared with ML models in SA of medication reviews. Results have indicated that DL models have promising results compared to traditional ML models6,33,34. Finally, in recent studies, the combination of DL models such as CNN-Gated Recurrent Unit (GRU) or bidirectional models such as Bidirectional Gated Recurrent Unit (Bi-GRU) have been suggested to analyze sentiments in the dataset of patients’ medication reviews29,37.
Ebrahimi et al.31 proposed a sentiment extraction recognition system using rule-based and SVM algorithms to recognize side effects in medication reviews. The medication reviews were hand-selected by a medical experiment for model training and testing. Their results have indicated that the SVM algorithm had significantly better performance than rule-based algorithms.
Gräßer et al.28 have performed both cross-domain and in-domain SA using LR to predict the sentiments of overall satisfaction, effectiveness, and side effects of the medications. They crawled Drugs.com and Druglib.com to obtain their data. In the cross-domain SA, using Drugs.com data, they achieved 70.06% for accuracy and 26.76% for kappa. The results for the side effects were 49.75% for accuracy and 25.88% for kappa.
Chen et al.30 have applied SA of the medication reviews using NB, DT, RF, and Ripper to positive, negative, and neutral labels. They collected their data by scraping the Hypertext Markup Language (HTML) file of Druglib.com. To solve the problem of many features, they employed Fuzzy-rough feature selection to reduce the data effectively. They used two techniques to determine the value of terms: the Bag of Words (BoW) and the Term Frequency-Inverse Document Frequency (TF-IDF). The best results were obtained using the BoW method. RF model, with 66.41% accuracy, was the most accurate model, and the average accuracy of the four implemented ML models was 63.85%.
Jiménez-Zafra et al.32 applied lexicon-based and supervised learning SA approaches to inform patients’ sentiments for both physician and medication purposes. They have collected their dataset in Spanish from internet forums. Their results have indicated that it is much more challenging to classify medication opinions from opinions related to doctors. Liu et al.39 have proposed a new feature extraction approach using position embeddings for SA of medication reviews. They implemented different feature extraction techniques with various ML models to classify sentiments and concluded that their proposed method had superior performance.
Colón-Ruiz et al.8 have used patients’ medication reviews to predict feelings about taking each medication. For this aim, they developed a CNN for classification. Their results indicated that precision, recall, and F1-Score values were better than classical classification methods such as SVM. Zhang et al.35 provided a dataset of adverse medication reactions and proposed a new DL architecture for its identification. Their results showed that their proposed model performance was slightly better than other models in detecting adverse medication reactions.
Basiri et al.6 have proposed two different methods using the traditional techniques of ML and DL to distinguish sentiments from patients’ medication reviews. They compared their best proposed method with traditional and DL models. According to their reported results, they increased accuracy and F1-Measure by 4%.
Jain et al.29 have developed DL models, including CNN, GRU, Multi-Instance Learning (MIL), and CNN-GRU, to identify the deceptive comments and sentiments from different datasets, including the medication review dataset. They used two MIL and Hierarchical architecture methods and claimed that they achieved better results using DL models compared to previous studies where traditional ML models were developed. In the SA of medication review dataset, CNN obtained the values of 76.8%, 0.77, and 0.77, GRU obtained the values of 76.3%, 0.76, and 0.76, MIL obtained the values of 78.2%, 0.78, and 0.78, and CNN-GRU obtained the values of 83.8%, 0.84, and 0.83 for accuracy, precision, and recall, respectively.
Han et al.37 have used two Bi-GRU neural networks and the attention mechanism to produce a bidirectional semantic representation of medication review. They have stated that their proposed approach can improve performance compared to other modern architectures for classifying the sentiments of medication review.
Our literature review shows that previous studies exclusively have used ML and DL models to predict patients’ sentiments in medication reviews. However, the previous studies were done without considering the interpretability and explainability issues of AI models, and different pre-trained word embeddings in the clinical domain were not considered for SA of medication reviews6,8,28,29,30,31,32,33,34,35,36,37,38,39. Furthermore, the ENS learning models have not been used in any of them.
Nevertheless, previous studies have used SA widely in non-medication fields, indicating its use in shopping, tourism, and social networks14,15,16,17,18,19,20,40. In recent years, the application of SA for medicine-related information in social networks has also been considered21,22,23,24,25,26,27. Furthermore, previous studies in medication SA have attempted to provide classical ML and DL methods to predict medication review sentiments6,8,28,29,30,31,32,33,34,35,36,37,38,39. For instance, recent systematic reviews have confirmed that SA is a promising tool in unexpected events such as the COVID-19 pandemic, which can be used to inform the public’s sentiments about the disease and its vaccination40,41.
According to recent systematic reviews6,8,28,29,30,31,32,33,34,35,36,37,38,39,42,43 and our literature review, existing gaps in the field of SA in medication have been identified. First, shared reviews about medications on the internet can be a beneficial source for SA that can help clinicians get insights about patients’ conditions as well as side effects42. Second, the creation of a special and comprehensive sentiment lexicon for medication reviews SA in the clinical domain, which was endorsed by clinicians and pharmacists, has not been addressed in previous studies6,8,12,28,29,30,31,32,33,34,35,36,37,38,39,43. Third, even though drugs.com is a rich dataset for SA analysis, there is no comprehensive study using this dataset6,8,28,29,30,31,32,33,34,35,36,37,38,39,42. Fourth, some studies have used meta-heuristics, optimization, and fuzzy methods19,30, but newly developed and improved meta-heuristics and optimization algorithms were not considered in the feature selection process and finding models’ optimal hyperparameters44,45,46. Fifth, no study has used various pre-trained word embeddings in the clinical domain for SA in medication reviews. Finally, no study considered explainable AI techniques to predict SA in the medication review dataset6,8,28,29,30,31,32,33,34,35,36,37,38,39,42,43.
Therefore, to address these gaps, in this study, a DL_ENS model has been proposed to comprehensively predict the analysis of patients’ sentiments in medication reviews28 with different scenarios. Also, different pre-trained word embeddings in the clinical domain were integrated into the proposed model to overcome the lack of a specific lexicon for the medication SA challenge. Furthermore, an explainable method of AI was used to increase the transparency and explainability of our proposed best model (DL_ENS). Addressing this issue can help clinicians to understand the decision-making process and increase their trust in the results.
The dataset of this study was extracted from drugs.com28, which is publicly available in the UCI Machine Learning “Drug Review Dataset (Drugs.com)” repository. The medication review dataset contains 215,063 patients’ sentiments (text) about the medication they used, along with a score from 1 to 10 (numerical) that patients have registered as well as the condition of the medications (text)28. The methodology diagram of this study is illustrated in Fig. 1.
Workflow diagram illustrating the steps performed in this study.
In this study, NumPy and NLTK libraries were used to perform preprocessing tasks on medication review texts. This stage had five steps as: (1) removing all incomplete records, (2) removing the redundant punctuation marks and characters from all texts, (3) converting uppercase letters to lowercase letters in all instances of the dataset, (4) deleting the stop words because they were frequently present in the texts and did not provide us with the helpful information in terms of performance, and (5) using Snowball Stemmer to remove the suffixes of the words in the dataset and find their roots. After fully preprocessing, 213,869 samples were remained.
After preprocessing phase, a clean and consistent dataset was created. ML and DL models are not able to work directly with texts. So, texts should be converted to vectors of numbers. This study utilized BoW and word embedding techniques to extract features from texts. BoW is a simple way to count the number of repetitions of words and an easy-to-use technique for converting texts to vectors for classification models47.
Word embedding is a new method to represent any word using a vector of numbers such that each number in the vector represents one latent feature of the word, and the vector represents different latent features of the word48. Word2Vec is a neural network-based word embedding technique. This method comprises three layers: input, hidden, and output. The Word2Vec method consists of two structures, Skip-Gram (SG) and Continuous BoW (CBOW)48. Furthermore, pre-trained word embedding, including Glove in the general domain, and PubMed, PMC, and combined PubMed and PMC in the clinical domain were considered in this study for DL models49,50.
Three scenarios were implemented in this study. In the first scenario, the scores of the medication reviews dataset were divided into two classes: Negative (for the scores less or equal to 5) and Positive (for the scores greater than 5). In the second scenario, the scores were divided into three classes: Negative (for the scores less than 5), Neutral (for the scores 5 and 6), and Positive (for the scores greater than 6). Eventually, in the third scenario, the dataset scores of this study were considered from one to ten for each medication review.
This study used Hold-Out cross-validation to split patients’ medication review dataset51. According to this method, the dataset was randomly divided into two training and testing sets, so 75% of the dataset was considered for training (160,093 samples) and 25% for testing (53,776 samples).
Nine common models of ML, DL, and ENS with different theoretical backgrounds, including KNN, DT, RF, Artificial Neural Network (ANN), Bidirectional Recurrent Neural Network (Bi-RNN), Bidirectional Long Short-Term Memory (Bi-LSTM), Bi-GRU, Machine ENS learning (ML_ENS), and DL_ENS52,53,54,55,56,57, were developed to predict patients’ sentiment and rate scores. All proposed ML algorithms in this study are described in detail in Appendix A.
DL algorithms depend on activation functions and loss functions during their learning process. By utilizing these functions and updating the weights, the models are trained to make predictions. Rectified Linear Unit (ReLU) is a type of activation function that is used in neural networks to introduce the property of non-linearity to them, and help them learn complex patterns and predict more accurately55. This function sets negative values of input to zero, while leaving positive values unchanged55. The mathematical equation of the ReLU activation function is as follows:
Where (:x) represents the input value.
Sigmoid is an activation function that takes any inputs and transforms them into output values in the range of 0 to 1 in neural networks55. The sigmoid function is commonly used in binary classification tasks55. The following equation shows how the sigmoid activation function works:
Where (:sigma:) represents the sigmoid function, and (:e) represents Euler’s number.
Softmax is an activation function that converts numbers or logic to probabilities. Softmax’s output is a vector of probabilities for the possible outcomes55. It is used to normalize the output of neural networks. Unlike the sigmoid activation function, it is commonly utilized for multivariate classification tasks55. The following equation shows how the softmax activation function works:
Where (:S) is softmax, (:overrightarrow{z}) devotes input vector, (:{e}^{{z}_{i}}) standard exponential function for input vector, (:K) shows the number of classes in multivariate classification, and (:{e}^{{z}_{j}}) means standard exponential function for output vector.
The loss function is a function that is calculated to evaluate the models’ performance in modeling the dataset55. In other words, it measures the difference between the predicted and actual target values55. The following equation represents how binary log loss is calculated:
Where (:{y}_{i}) shows actual values, and (:{widehat{y}}_{i}) shows model predictions.
Another loss function that is used for multivariate classification is categorical cross-entropy loss55, which is calculated as follows:
Where (:{y}_{i}) shows actual values, and (:{widehat{y}}_{i}:)shows model predictions55.
RNN is a type of ANN which used in text, speech, and sequential data processing54,55. Unlike feed-forward networks, RNNs have a feedback layer where the output of the network and the next input are fed back to the network55. RNNs have internal memory, so they can remember their previous input and use their memory to process sequential inputs. Long Short-Term Memory (LSTM) and GRU are among the RNN algorithms in which the output of the previous layers is used as input to the subsequent layers54. LSTM and GRU, in their architecture, solve the vanishing gradient problem that occurs in RNN54,55. Bi-RNN, Bi-LSTM, and Bi-GRU algorithms have a two-way architecture55. These three algorithms move and learn in two directions (forward and backward) in a progressive and regressive way55.
The output of Bi-RNN is:
Where
That (:{x}_{t}) denotes the input vector at the time (:t), (:{y}_{t}) is the output vector at the time (:t),(::{h}_{t}) is the hidden layer at the time (:t), (:f) means forward, (:b) means backward, and (:{W}_{y}), (:{W}_{h}), and (:{W}_{x}) denote the weight matrices that connect the hidden layer to the output layer, the hidden layer to the hidden layer, and the input layer to the hidden layer, respectively. (:{b}_{y}) and (:{b}_{h}) are the bias vectors of the output and hidden layer, respectively55.
In the following, the calculation process of Bi-LSTM is explained:
.
Equations (9–14) are the equations of the forgotten gate, input gate, current state of the cell, memory unit status value, output gate, and hidden gate, respectively. The (:b) and (:W) denote the bias vector and weight coefficient matrix55. (:sigma:) shows the sigmoid activation function55.(::{x}_{t}) denotes the input vector at the time (:t) and (:{h}_{t}) is the hidden layer at the time (:t)55. The output of Bi-LSTM is:
Where
That (:{y}_{t}) is the output vector at the time (:t),(::f) means forward, (:b) means backward, and (:V), (:W), and (:U) denote the weight matrices that connect the hidden layer to the output layer, hidden layer to hidden layer, and input layer to hidden layer, respectively54.
The calculation process of Bi-GRU is:
Where (:W) is the weight matrix, (:{z}_{t}) shows update gate, (:{r}_{t}:)represents the reset gate, (:{stackrel{sim}{h}}_{t}) shows reset memory, and (:{h}_{t}) shows new memory. (:{x}_{t}) denotes the input vector at the time (:t), and (:b) is the bias vector55. The output of Bi-GRU is:
Where
That (:GRU) is the traditional GRU computing process, (:f) and (:b) mean forward and backward, respectively, and (:{b}_{t}) is the bias vector at the time (:t).
Supposing (:h) is equal to Eq. (6) for Bi-RNN, Eq. (15) for Bi-LSTM, and Eq. (22) for Bi-GRU, and the parameters are: the number of these models units is (:150), the number of units in the first fully connected layer is (:n=128), and the number of units in the second fully connected layer is (:z), given a single time step input:
Where Eqs. (25, 26) are the equations of the first fully connected layer with ReLU activation and the second fully connected layer, respectively. (:{f}_{i}) could be sigmoid as Eq. (2) for the first approach, and it could be softmax as Eq. (3) for the second and third approaches. The (:{b}_{1}) and (:{b}_{2}) denote the bias vectors and (:{W}_{1}) and (:{W}_{2}) denote weight coefficient matrices.
By knowing (:t=1), and (:{f}_{i}) is sigmoid, we have these proposed algorithms:
Also, if (:t=1), and (:{f}_{i}) is softmax, we have these proposed algorithms:
ENS learning is an AI technique to increase the model’s power in estimating data output, which uses several models in combination and simultaneously to make decisions56. One of the ENS learning methods is the voting method, in which decisions are made based on the votes of the models, and it includes two approaches, hard and soft voting52. In hard voting, the choice of target is based on the maximum number of votes the models have given to the output56. In soft voting, the target selection is based on the highest joint probability that the models had over the output52,56. In this paper, the hard voting method is used to develop two ENS models of ML_ENS and DL_ENS. The equation of hard voting is represented as follows:
Where (:t=:{KNN,DT,:RF,:ANN}) in ML_ENS model and (:t={Bi-RNN,:Bi-LSTM,::Bi-GRU}) in DL_ENS model, (::j={Negative,:Positive}) in the first scenario, (::j={Negative,::Neutral:,Positive}) in the second scenario, and (::j={One,:Two,:Three,:Four,:Five,:Six,:Seven,:Eight,:Nine,:Ten}) in the third scenario. (:T)represents the number of models, and (:C) represents the number of classes. Nonetheless, the mathematical forms of the ML_ENS and DL_ENS models, according to the aforementioned sentences, determine the target class in each approach by voting from all the proposed algorithms.
In this study, Sklearn and TensorFlow libraries were used for implementation. Grid Search was applied to find the best values of hyperparameters. This method searches and evaluates the grid in which hyperparameters and their values are specified and determines the best hyperparameter values for each model57. The best selected hyperparameters for proposed models are shown in Table 2. Once the best hyperparameters were identified for each model, these tuned models were chosen to create ML_ENS and DL_ENS models. The ENS approaches then combined the predictions from these optimized models. The aggregated votes of these tuned models determined the final prediction of each ENS model. This process ensured that the ensemble models benefited from the strengths of each individually optimized model to improve overall prediction performance. Furthermore, weighted loss functions were considered to address the class imbalance and ensure that the model paid more attention to the minority class during training. Specifically, a weight was assigned to each class based on its frequency in the dataset, so that underrepresented classes were given more importance in the optimization process. This approach helps mitigate the negative effects of class imbalance on model performance. We developed our algorithm on a server with 32 GB of RAM, Intel E5-2650 CPU, and 4 GB memory by GPU Nvidia GTX 1650.
The following evaluation criteria were considered to evaluate the performance of the proposed models58:
TP, TN, FP, and FN are True Positive, True Negative, False Positive, and False Negative. These are components of the confusion matrix59. Moreover, the Area Under Curve (AUC) metric was used to estimate the performance of the best model as it often provides a better evaluation of performance than the accuracy metric60.
LIME is an interpretable and explainable method for AI black box models59,61. The LIME is a simple but powerful way to interpret and explain models’ decision-making processes59,61. This method considers the most influential features to explain how the model predicts. LIME locally approximates the prediction by forming a disturbance in the input around the class so that when a linear approximation is reached, it explains and justifies the model’s behavior and performance61.
SA is a prominent component of natural language processing applications. One field that has attracted researchers’ attention is using SA to evaluate medications and their impacts on patients. The current study used the medication review dataset to predict patients’ sentiments and rate scores based on their responses to the medications in the proposed scenarios. For this purpose, four ML models, three DL models, and two ENS models with a hard voting approach were developed.
The patients’ medication reviews database was preprocessed to achieve the best performance. So, redundant samples were removed, and their possible negative impact and bias on the models’ performance were avoided as much as possible. At the end of the preprocessing, 213,869 patient comments remained as the main dataset for this study. Figure 2 shows the most common patient conditions with more than two thousand samples in this dataset. According to Fig. 2, birth control, depression, pain, anxiety, and acne are the five most common conditions. Also, Fig. 3 shows the share of each score in this dataset. According to Fig. 3, the scores of 10, 9, 1, 8, and 7 have the largest share in this dataset.
The most common patient conditions in the dataset.
Chart showing the distribution of each score in the dataset.
According to Table 3, this study has considered different scenarios for predicting sentiment classes and rate scores. The reason for these different scenarios is the imbalanced number of samples for each class. Especially in the second scenario, the number of samples of the neutral class is less than in other classes. Therefore, the third scenario has been created with ten classes to improve the performance.
Based on experiments, we selected a BoW vector size of 20,000 for ML models among values (10,000–25,000). Also, the CBOW method from the gensim library in Python was selected by the dimensionality of the 15,000 words number size for DL models among values (1,000–25,000). After completing the preprocessing and selecting the best hyperparameters, our proposed models were executed and tested. Then, models were evaluated based on the evaluation criteria (Eqs. (34–37)) considered in Sect. “Evaluation of models”.
As mentioned, patients’ sentiments were predicted based on three scenarios. Accuracy, precision, recall, and F1-Score were calculated for all models. Table 4 shows the results of all proposed models in the first scenario (two classes). As seen, among traditional ML and DL models, RF and Bi-GRU had the best performance, respectively. Moreover, the DL_ENS model outperforms all ML, DL, and ENS models proposed in this study.
Results of all models in the second scenario (three classes) are presented in Table 5. As shown RF model had the best performance among the ML models. Among the proposed DL models, the Bi-GRU model had better performance. DL_ENS model also had the highest achievement among all proposed models. The results of the models proposed in this study in the third scenario (ten classes) are presented in Table 6. RF model performed better than other ML models. Among the proposed DL models, the Bi-GRU model had the best performance. Finally, the best proposed model (DL_ENS) showed better results than all ML, DL, and ML_ENS models.
According to Tables 4, 5 and 6, the results of the DL_ENS model in all scenarios clearly show that this model (DL_ENS) has outperformed other implemented models. Therefore, the DL_ENS model was utilized and tested for further investigation using general and clinical pre-trained word embeddings. The results of these experiments for the first, second, and third scenarios are presented in Tables 7, 8 and 9. It is worth noting that clinical pre-trained word embedding (combined PubMed and PMC) had superior performance in the proposed best model and was able to increase its accuracy and F1-score values. Also, clinical pre-trained word embeddings had more positive impacts on the models’ performance than general domain pre-trained word embeddings. Furthermore, the computation time of the best model of this study on different pre-trained word embeddings for various scenarios is shown in Table 10. In addition, the proposed best model was compared with previous studies6,8,28,29,30 on the same dataset, and the comparison is presented in Table 11. This table shows that DL_ENS outperformed other previously reported models6,8,28,29,30.
Figures 4 and 5 show the AUC diagram and confusion matrices for all scenarios of the proposed DL_ENS model, respectively. As shown in Figs. 6, 7 and 8, and Figs. S1-S6 in Appendix A, three reviews from the test dataset were selected randomly to interpret the decisions of the best model in all scenarios. Therefore, this study utilized LIME, which explains the decisions of these random medication reviews.
The AUC diagrams illustrating the performance of the best model.
Confusion matrices illustrating the classification performance of the best model.
Explanation of decisions made by the best model in the first scenario.
Explanation of decisions made by the best model in the second scenario.
Explanation of decisions made by the best model in the third scenario.
In this study, three different scenarios are presented to predict patients’ SA about medication. The first and second scenarios are the most common modes for predicting SA, as used in most studies6,8,28,29,30. In the first scenario, scores less than 5 were classified as negative, and scores 5 and above were classified as positive. In the second scenario, scores less than 5 were classified as negative, scores 5 and 6 as neutral, and scores above 6 as positive. To identify patients’ sentiments more accurately towards medications, a third scenario considered ten classes, with scores ranging from 1 to 10 based on what patients recorded for each medication.
Comparing the results of the best proposed model in Tables 4, 5 and 6 with Tables 7, 8 and 9, in the first scenario, the accuracy value is increased by 1.57%, and the value of F1-Score is increased by 1.58%. Also, in the second scenario, the accuracy value is increased by 1.23%, and the F1-Score value is increased by 1.15%. Finally, in the third scenario, the accuracy value is increased by 0.36%, and the F1-Score value is increased by 0.83%. The best proposed model with pre-trained word embedding (a combination of PubMed and PMC) obtained the best result in all scenarios of this study.
According to Table 10, the computing time of the best-proposed model (with PubMed and PMC pre-trained word embedding) is comparable with that of other word embeddings. It is important to note that the computational costs across different model configurations can vary, which is crucial for assessing the practicality of the models in real-world applications. While the PubMed and PMC combination achieves the highest performance, its computation time is slightly longer than other word embeddings. For instance, the runtime for the third scenario with the PubMed and PMC embedding is 18,952.50 s, while Word2Vec and GloVe require 18,055.00 and 18,495.00 s, respectively. These time differences should be considered when evaluating the trade-off between model accuracy and computational efficiency, especially in resource-constrained environments.
Based on62, the results provided in Figs. 4 and 5 indicate outstanding performance in the first and second scenarios and acceptable performance in the third scenario for the proposed best model using pre-trained PubMed and PMC word embedding models.
Medication SA can provide insight into the effectiveness of a medication and treatment process. Nonetheless, detecting and predicting patients’ sentiments towards medication is very complicated32, but by doing so, we can identify views about the medications, their unreliability, and their effectiveness; consequently, it can be used in clinical decision-making12.
Transparency is one of the main issues for developing AI models in clinical practices, and most AI models are considered as a black box where the logic of decision-making is not easy to understand and follow, especially for clinicians61. To address this, we used the LIME technique specifically designed to interpret complex machine learning models by approximating their decision-making process locally around each prediction. LIME works by perturbing the input data (words in medication reviews) and observing how these perturbations affect the model’s predictions. This method allows us to identify each prediction’s most influential features (words).
As shown in Figs. 6, 7 and 8 and Figs. S1-S6 in Appendix A, LIME was applied to explain the predictions made by the best-performing model. For each instance, LIME generated a local surrogate model that approximates the decision boundary of the black-box model in the vicinity of the specific prediction. By calculating the importance of each word for sentiment classification and rate scoring, we could identify the words that had the most significant impact on the model’s output. This step-by-step explanation of model decisions helps to uncover the rationale behind sentiment predictions, making them more transparent.
The output of LIME highlights the most influential words and provides a confidence score for each class based on the sum of the weighted probabilities of these words. This transparency level enables clinicians to better understand the reasoning behind the model’s predictions, increasing the trustworthiness and potential for adopting AI-driven tools in clinical workflows. By providing clear explanations of how sentiment analysis is performed, clinicians can use these models more confidently, ultimately aiding in more informed patient-care decision-making.
This research aimed to extend previous work by focusing on SA of patient medication reviews, which presented unique challenges requiring specialized models32. Compared to the models proposed in previous studies6,8,15,16,17,18,19,24,25,28,29,30,31,32,35,37,39, both for non-medication and medication reviews datasets, and as shown in Table 11, the performance of the DL_ENS model was superior to traditional DL and ML models. ENS learning mitigates errors, reduces overfitting, and increases accuracy and robustness by leveraging the strengths of different models63,64. Additionally, pre-trained word embeddings enhanced model performance and generalizability by capturing semantic and syntactic meanings from large datasets, as recommended by the study65.
The absence of ENS models and the use of pre-trained word embeddings in previous studies6,8,15,16,17,18,19,24,25,28,29,30,31,32,35,37,39, as well as in other proposed models of this study, has resulted in lower performance compared to the best proposed model. In this study, the DL_ENS model, combined with pre-trained word embeddings (from PubMed and PMC), leverages the advantages of DL models, ENS learning, and pre-trained word embeddings, showing promising results in all scenarios.
Several reasons caused this study’s improvement compared to previous studies: (1) comprehensive preprocessing was applied to the dataset to prevent errors and bias in the model prediction; (2) a systematic way to find the best hyperparameters was deployed; and (3) the recommended DL_ENS model in SA63 was used along with pre-trained word embedding in the clinical domain. So, in this study, PubMed and PMC were chosen as the pre-trained word embeddings due to their strong relevance to the clinical domain. These embeddings are trained on extensive biomedical corpora, allowing them to capture medical terminology and language patterns specific to healthcare. This choice enhances the model’s ability to understand and process sentiment in medication reviews, ensuring more accurate and contextually relevant predictions in the healthcare setting.
The development of these models, such as the proposed DL_ENS model, can serve as an auxiliary tool for clinicians and patients in the health and pharmaceutical systems to predict the sentiment of used medications. Initially, we developed several models based on a patient sentiment dataset about medications from the drugs.com website, which is for patients in the United States. The best model proposed in this study can also be tested and retrained on datasets from other countries as an external validation to enhance its effectiveness. This model can be integrated as a plug-in tool with software systems for registering and managing patient medications. Analyzing the collected sentiments can help clinicians prescribe more suitable medications with fewer side effects. This integration means patients and clinicians do not need to understand the model in detail, as it can be seamlessly incorporated into their routine software applications. Furthermore, the findings and detailed implementation of this study can guide the development of more advanced and comprehensive models by experts in AI applications in medicine. Finally, integrating the DL_ENS model in clinical settings raises important ethical considerations, particularly regarding the use of patient-generated data for model training and deployment. It is important to note that the dataset used in this study contains no patient information, as all records have been anonymized to ensure privacy and confidentiality. Patient consent and transparency are essential principles in this approach, as no identifiable data were involved in the analysis. Furthermore, as the model is integrated into clinical decision-making, it is crucial to remain free from biases that could affect treatment outcomes, especially for underrepresented patient groups. Continuous monitoring of the model’s real-world performance will be necessary to address any ethical concerns, ensuring it serves patients’ best interests.
However, our study has a few limitations. Firstly, in all scenarios proposed in this study, the number of instances of considered classes was not the same due to the dataset’s imbalanced number of reviews for different classes. Secondly, we could not develop more models due to limited resources. Additionally, there was no other suitable dataset similar to the one used, with a high number of samples, for external validation to verify the generalizability of the developed models. Finally, the DL_ENS model has its own limitations, such as higher computational costs and potential challenges in real-time deployment, which could impact its practical application in clinical settings. These factors may require further optimization to balance efficiency for practical use in healthcare environments.
In this study, the proposed DL_ENS model, a deep ENS model with PubMed and PMC pre-trained word embedding in all implemented scenarios, obtained the best results. Additionally, our findings show that pre-trained word embedding in the clinical domain has a better impact on the models’ performance rather than pre-trained word embedding in the general domain. Moreover, our developed model performs better than previous studies for SA on the drugs.com dataset. Our study also considers the explainability and interpretability of results by utilizing the LIME technique. This technique can help clinicians see the logic behind the decisions and increase trust in the results. As a result, this technique can also increase the chance of using our best model as a clinical decision support system in real practice.
This study’s proposed technique can enhance transparency and facilitate the integration of our DL_ENS model as a clinical decision support tool, which could be added as a plug-in to existing healthcare systems. This integration would allow clinicians to use sentiment insights from patient reviews to assist in medication selection, treatment planning, and patient counseling. The model can contribute to more personalized and effective care strategies by providing actionable insights into patient satisfaction and side effects, ultimately improving patient outcomes.
In the future, we plan to develop more models to increase the accuracy of SA in predicting patients’ sentiments. In addition, we intend to explore multi-modal approaches by integrating both text and voice data for sentiment analysis, testing the performance of such models in clinical practice. This extension could provide a more comprehensive understanding of patient sentiments by incorporating different forms of communication, potentially enhancing the model’s robustness and applicability in real-world healthcare settings.
The dataset analyzed during the current study is available publicly in the “Drug Review Dataset (Drugs.com)” repository, https://archive.ics.uci.edu/dataset/462.
Cambria, E., Wang, H. & White, B. Guest editorial: Big social data analysis. Knowl. Based Syst. 69, 1–2 (2014).
Article Google Scholar
Budennyy, S., Kazakov, A., Kovtun, E. & Zhukov, L. New drugs and stock market: A machine learning framework for predicting pharma market reaction to clinical trial announcements. Sci. Rep. 13, 12817 (2023).
Article ADS CAS PubMed PubMed Central Google Scholar
Grassi, M., Cambria, E., Hussain, A. & Piazza, F. Sentic web: A new paradigm for managing social media affective information. Cognit Comput. 3, 480–489 (2011).
Article Google Scholar
Chintalapudi, N., Battineni, G., Canio, M. D., Sagaro, G. G. & Amenta, F. Text mining with sentiment analysis on seafarers’ medical documents. Int. J. Inform. Manage. Data Insights. 1, 100005 (2021).
Google Scholar
Blanco, A., Casillas, A., Pérez, A. & de Diaz, A. Multi-label clinical document classification: Impact of label-density. Expert Syst. Appl. 138, 112835 (2019).
Article Google Scholar
Basiri, M. E., Abdar, M., Cifci, M. A., Nemati, S. & Acharya, U. R. A novel method for sentiment classification of drug reviews using fusion of deep and machine learning techniques. Knowl. Based Syst. 198, 105949 (2020).
Article Google Scholar
Hiremath, B. N. & Patil, M. M. Enhancing optimized personalized therapy in clinical decision support system using natural language processing. J King Saud Univ. – Comput Inf. Sci. 34, 2840–2848 (2022).
Google Scholar
Colón-Ruiz, C., Segura-Bedmar, I. & Martínez, P. Análisis de Sentimiento en el dominio salud: Analizando comentarios sobre fármacos. Procesamiento del. Lenguaje Nat. 63, 15–22 (2019).
Google Scholar
Sutphin, C., Lee, K., Yepes, A. J., Uzuner, Ö. & McInnes, B. T. Adverse drug event detection using reason assignments in FDA drug labels. J. Biomed. Inf. 110, 103552 (2020).
Article Google Scholar
Snyder, C. F. et al. The role of informatics in promoting patient-centered care. Cancer J. 17, 211–218 (2011).
Article PubMed PubMed Central Google Scholar
Liu, Z. et al. Entity recognition from clinical texts via recurrent neural network. BMC Med. Inf. Decis. Mak 17, (2017).
Denecke, K. & Deng, Y. Sentiment analysis in medical settings: New opportunities and challenges. Artif. Intell. Med. 64, 17–27 (2015).
Article PubMed Google Scholar
Cambria, E., Poria, S., Gelbukh, A. & Thelwall, M. Sentiment analysis is a big suitcase. IEEE Intell. Syst. 32, 74–80 (2017).
Article Google Scholar
Chen, J. et al. A classified feature representation three-way decision model for sentiment analysis. Appl. Intell. 52, 7995–8007 (2022).
Article Google Scholar
Wu, D. C., Zhong, S., Qiu, R. T. R. & Wu, J. Are customer reviews just reviews? Hotel forecasting using sentiment analysis. Tourism Econ. 28, 795–816 (2022).
Article Google Scholar
Sarsam, S. M., Al-Samarraie, H., Alzahrani, A. I., Alnumay, W. & Smith A. P. A lexicon-based approach to detecting suicide-related messages on Twitter. Biomed. Signal. Process. Control. 65, 102355 (2021).
Article Google Scholar
Gao, Z., Li, Z., Luo, J. & Li, X. Short Text Aspect-Based Sentiment Analysis Based on CNN + BiGRU. Applied Sciences 12, (2022).
Yu, W., Cui, F. & Hou, Z. The evolution of consumers’ demand for hotels under the public health crisis: Opinion mining from online reviews. Curr. Issues Tourism. 26, 1974–1990 (2023).
Article Google Scholar
Yildirim, G. A novel grid-based many-objective swarm intelligence approach for sentiment analysis in social media. Neurocomputing 503, 173–188 (2022).
Article Google Scholar
AL-Smadi, M., Hammad, M. M., Al-Zboon, S. A., AL-Tawalbeh, S. & Cambria, E. Gated recurrent unit with multilingual universal sentence encoder for Arabic aspect-based sentiment analysis. Knowl. Based Syst. 261, 107540 (2023).
Article Google Scholar
Singh, C., Imam, T., Wibowo, S. & Grandhi, S. A deep learning approach for sentiment analysis of COVID-19 reviews. Appl. Sci. (Basel). 12, 3709 (2022).
Article CAS Google Scholar
Marcec, R. & Likic, R. Using Twitter for sentiment analysis towards AstraZeneca/Oxford, Pfizer/BioNTech and Moderna COVID-19 vaccines. Postgrad. Med. J. 98, 544–550 (2022).
Article PubMed Google Scholar
Chinnasamy, P. et al. COVID-19 vaccine sentiment analysis using public opinions on Twitter. Mater. Today. 64, 448–451 (2022).
CAS Google Scholar
Tang, J. E. et al. How are patients reviewing spine surgeons online? A Sentiment Analysis of physician review website written comments. Global Spine J. 13, 2107–2114 (2023).
Article PubMed Google Scholar
Chandra, R. & Krishna, A. COVID-19 sentiment analysis via deep learning during the rise of novel cases. PLoS One. 16, e0255615 (2021).
Article CAS PubMed PubMed Central Google Scholar
Zulfiker, M. S., Kabir, N., Biswas, A. A., Zulfiker, S. & Uddin, M. S. Analyzing the public sentiment on COVID-19 vaccination in social media: Bangladesh context. Array (N Y). 15, 100204 (2022).
Article PubMed Google Scholar
Alam, K. N. et al. Deep learning-based sentiment analysis of COVID-19 vaccination responses from Twitter data. Comput. Math. Methods Med. 4321131 (2021). (2021).
Gräßer, F., Kallumadi, S., Malberg, H. & Zaunseder, S. Aspect-based sentiment analysis of drug reviews applying cross-domain and cross-data learning. in Proceedings of the International Conference on Digital Health (ACM, New York, NY, USA, 2018). (ACM, New York, NY, USA, 2018). (2018).
Jain, N., Kumar, A., Singh, S., Singh, C. & Tripathi, S. Deceptive reviews detection using deep learning techniques. in Natural Language Processing and Information Systems 79–91Springer International Publishing, Cham, (2019).
Chen, T. et al. Sentiment classification of drug reviews using fuzzy-rough feature selection. in. IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) (IEEE, 2019). (2019).
Ebrahimi, M., Yazdavar, A. H., Salim, N. & Eltyeb, S. Recognition of side effects as implicit-opinion words in drug reviews. Online Inf. Rev. 40, 1018–1032 (2016).
Article Google Scholar
Jiménez-Zafra, S. M., Martín-Valdivia, M. T. & Molina-González, M. D. Ureña-López, L. A. How do we talk about doctors and drugs? Sentiment analysis in forums expressing opinions for medical domain. Artif. Intell. Med. 93, 50–57 (2019).
Article PubMed Google Scholar
Youbi, F. & Settouti, N. Analysis of machine learning and deep learning frameworks for opinion mining on drug reviews. Comput. J. 65, 2470–2483 (2022).
Article Google Scholar
Ru, B., Li, D., Hu, Y. & Yao, L. Serendipity-A machine-learning application for mining serendipitous drug usage from social media. IEEE Trans. Nanobiosci. 18, 324–334 (2019).
Article Google Scholar
Zhang, M. & Geng, G. Adverse drug event detection using a weakly supervised convolutional neural network and recurrent neural network model. Information (Basel) 10, 276 (2019).
Colón-Ruiz, C. & Segura-Bedmar, I. Comparing deep learning architectures for sentiment analysis on drug reviews. J. Biomed. Inf. 110, 103539 (2020).
Article Google Scholar
Han, Y., Liu, M. & Jing, W. Aspect-level drug reviews sentiment analysis based on double BiGRU and knowledge transfer. IEEE Access. 8, 21314–21325 (2020).
Article Google Scholar
Hossain, M. D., Azam, M. S., Ali, M. J. & Sabit, H. Drugs rating generation and recommendation from sentiment analysis of drug reviews using machine learning. in 2020 Emerging Technology in Computing, Communication and Electronics (ETCCE) (IEEE, (2020).
Liu, S. & Lee, I. Extracting features with medical sentiment lexicon and position encoding for drug reviews. Health Inf. Sci. Syst. 7, 11 (2019).
Article PubMed PubMed Central Google Scholar
Xu, Q. A., Chang, V. & Jayne, C. A systematic review of social media-based sentiment analysis: Emerging trends and challenges. Decis. Analytics J. 3, 100073 (2022).
Article Google Scholar
Cascini, F. et al. Social media and attitudes towards a COVID-19 vaccination: A systematic review of the literature. EClinicalMedicine 48, 101454 (2022).
Article PubMed PubMed Central Google Scholar
Zunic, A., Corcoran, P. & Spasic, I. Sentiment analysis in health and well-being: Systematic review. JMIR Med. Inf. 8, e16023 (2020).
Article Google Scholar
Pilipiec, P., Liwicki, M. & Bota, A. Using machine learning for pharmacovigilance: A systematic review. Pharmaceutics 14, 266 (2022).
Article PubMed PubMed Central Google Scholar
El-kenawy, E. S. M. et al. Greylag Goose Optimization: Nature-inspired optimization algorithm. Expert Syst. Appl. 238, 122147 (2024).
Article Google Scholar
El-kenawy, E. S. M. et al. Football Optimization Algorithm (FbOA): A novel metaheuristic inspired by team strategy dynamics. J. Artif. Intell. Metaheuristics. 8, 21–38 (2024).
Article Google Scholar
Abdollahzadeh, B. et al. Puma optimizer (PO): A novel metaheuristic optimization algorithm and its application in machine learning. Cluster Comput. 27, 5235–5283 (2024).
Article Google Scholar
Zhang, Y., Jin, R. & Zhou, Z. H. Understanding bag-of-words model: A statistical framework. Int. J. Mach. Learn. Cybern. 1, 43–52 (2010).
Article Google Scholar
Serrano-Guerrero, J., Bani-Doumi, M., Romero, F. P. & Olivas, J. A. Understanding what patients think about hospitals: A deep learning approach for detecting emotions in patient opinions. Artif. Intell. Med. 128, 102298 (2022).
Article PubMed Google Scholar
Pennington, J., Socher, R. & Manning, C. Glove: Global vectors for word representation. in Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP) (Association for Computational Linguistics, Stroudsburg, PA, USA, 2014). (Association for Computational Linguistics, Stroudsburg, PA, USA, 2014). (2014).
Moen, S. P. & Ananiadou, T. S. Distributional semantics resources for biomedical text processing. in Proceedings of LBM 39–44 (2013).
Shortliffe, E. H. & Sepúlveda, M. J. Clinical decision support in the era of artificial intelligence. JAMA 320, 2199–2200 (2018).
Article PubMed Google Scholar
Alzubi, J., Nayyar, A. & Kumar, A. Machine learning from theory to algorithms: An overview. J. Phys. Conf. Ser. 1142, 012012 (2018).
Article Google Scholar
Alpaydin, E. Introduction to Machine Learning (MIT Press, 2020).
Islam, M. S. & Ghani, N. A. A novel BiGRUBiLSTM model for multilevel sentiment analysis using deep neural network with BiGRU-BiLSTM. in Lecture Notes in Electrical Engineering 403–414Springer Singapore, Singapore, (2022).
Goodfellow, I., Bengio, Y. & Courville, A. Deep Learning (MIT Press, 2016).
Alsulami, B., Almalawi, A. & Fahad, A. Toward an efficient automatic self-augmentation labeling tool for intrusion detection based on a semi-supervised approach. Appl. Sci. (Basel). 12, 7189 (2022).
Article CAS Google Scholar
Pedregosa, F. et al. Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).
MathSciNet Google Scholar
Hossin, M. & Sulaiman, M. N. A review on evaluation metrics for data classification evaluations. Int. J. Data Min. Knowl. Manag. Process 5, (2015).
Babaei Rikan, S. et al. Survival prediction of glioblastoma patients using modern deep learning and machine learning techniques. Sci. Rep. 14, 2371 (2024).
Article ADS CAS PubMed PubMed Central Google Scholar
Ling, C. X., Huang, J., Zhang, H. & AUC A better measure than accuracy in comparing learning algorithms. in Advances in Artificial Intelligence, 329–341 (Springer 2003).
Linardatos, P., Papastefanopoulos, V., Kotsiantis, S. & Explainable, A. I. A review of machine learning interpretability methods. Entropy (Basel). 23, 18 (2020).
Article ADS PubMed Google Scholar
Mandrekar, J. N. Receiver operating characteristic curve in diagnostic test assessment. J. Thorac. Oncol. 5, 1315–1316 (2010).
Article PubMed Google Scholar
Kazmaier, J. & van Vuuren, J. H. The power of ensemble learning in sentiment analysis. Expert Syst. Appl. 187, 115819 (2022).
Article Google Scholar
Baradaran, R. & Amirkhani, H. Ensemble learning-based approach for improving generalization capability of machine reading comprehension systems. Neurocomputing 466, 229–242 (2021).
Article Google Scholar
Rezaeinia, S. M., Rahmani, R., Ghodsi, A. & Veisi, H. Sentiment analysis based on improved pre-trained word embeddings. Expert Syst. Appl. 117, 139–147 (2019).
Article Google Scholar
Download references
There is no funding attached to this study.
Amir Sorayaie Azar and Samin Babaei Rikan contributed equally to this work
SDU Health Informatics and Technology, The Maersk Mc-Kinney Moller Institute, University of Southern Denmark, Odense, Denmark
Amir Sorayaie Azar, Amin Naemi, Jamshid Bagherzadeh Mohasefi & Uffe Kock Wiil
Department of Computer Engineering, Urmia University, Urmia, Iran
Amir Sorayaie Azar, Samin Babaei Rikan & Jamshid Bagherzadeh Mohasefi
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
A.S.A., S.B.R., A.N., J.B.M., and U.K.W. were involved in the conception and design of this study. A.S.A. and S.B.R. prepared the dataset and performed the analysis. A.S.A., S.B.R., A.N., J.B.M., and U.K.W. interpreted the results. A.S.A. and S.B.R. drafted the manuscript, and all authors (A.S.A., S.B.R., A.N., J.B.M., and U.K.W.) contributed to writing the final draft and prepared the final manuscript. All authors read and approved by the final manuscript.
Correspondence to Jamshid Bagherzadeh Mohasefi.
The authors declare no competing interests.
Not applicable.
Not applicable.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Below is the link to the electronic supplementary material.
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
Reprints and permissions
Sorayaie Azar, A., Babaei Rikan, S., Naemi, A. et al. Predicting patients’ sentiments about medications using artificial intelligence techniques. Sci Rep 14, 31928 (2024). https://doi.org/10.1038/s41598-024-83222-9
Download citation
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-024-83222-9
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative
Advertisement
© 2024 Springer Nature Limited
Sign up for the Nature Briefing: Translational Research newsletter — top stories in biotechnology, drug discovery and pharma.
Français 
English