| Fortunately, the new developed platforms (e.g., DataCamp), and tools like tensor flow and keras making it easier to use deep learning algorithms which were built-in as python library. Although the barrier has been lower to access the door, it is still challenging to optimize / re-tune the model to acquire best predictions. Although deep leaning require larger amount of training data and computational power comparing to other typical machine learning methods (such as, random forests (RF), support vector machine (SVM) k-nearest neighbors (kNN)), their accuracy evaluation methods are similar. For the classification model of deep learning or machine learning, Overall classification performance can be quantified by different metrics: TNR = true negative rate (also specificity), TPR = true positive rate (also sensitivity, recall), precision, accuracy, area under the receiver operating characteristic (AUROC) curve, PRC = precision-recall curve. For all of the above metrics, values closer to 1 indicate increasingly optimal performance. L is the positive class. TP = true positive, FP = false positive, TN = true negative, FN = false negative. | Fortunately, the new developed platforms (e.g., DataCamp), and tools like tensor flow and keras making it easier to use deep learning algorithms which were built-in as python library. Although the barrier has been lower to access the door, it is still challenging to optimize / re-tune the model to acquire best predictions. Although deep leaning require larger amount of training data and computational power comparing to other typical machine learning methods (such as, random forests (RF), support vector machine (SVM) k-nearest neighbors (kNN)), their accuracy evaluation methods are similar. For the classification model of deep learning or machine learning, Overall classification performance can be quantified by different metrics: TNR = true negative rate (also specificity), TPR = true positive rate (also sensitivity, recall), precision, accuracy, area under the receiver operating characteristic (AUROC) curve, PRC = precision-recall curve. For all of the above metrics, values closer to 1 indicate increasingly optimal performance. L is the positive class. TP = true positive, FP = false positive, TN = true negative, FN = false negative. |
| For the input data set may be split into a Training Set (usually 75–90% of the input data) and a Test Set. When a Test Set cannot be made, cross validation (CV) may be sufficient to estimate the algorithm’s error in classification of test instances. Here, five-fold CV is shown. In each fold, 20% of the input training data is randomly chosen to be the Validation Set. The model is trained using the training data, and its performance on new instances is determined through the Validation Set. Once the benchmark dataset has been spited into test data and train data, they will be fitted in different models to evaluate the performance. Since the data points in the test set may not be representative of the model's ability to generalize to unseen data. To combat this dependence on what is essentially an arbitrary split, we use a technique called cross-validation. | For the input data set may be split into a Training Set (usually 75–90% of the input data) and a Test Set. When a Test Set cannot be made, cross validation (CV) may be sufficient to estimate the algorithm’s error in classification of test instances. Here, five-fold CV is shown. In each fold, 20% of the input training data is randomly chosen to be the Validation Set. The model is trained using the training data, and its performance on new instances is determined through the Validation Set. Once the benchmark dataset has been spited into test data and train data, they will be fitted in different models to evaluate the performance. Since the data points in the test set may not be representative of the model's ability to generalize to unseen data. To combat this dependence on what is essentially an arbitrary split, we use a technique called cross-validation. |
