Differences

This shows you the differences between two versions of the page.

--- bioinformatics_essay36 [2023/08/16 12:50] – 24.114.43.194
+++ bioinformatics_essay36 [2023/08/16 12:52] (current) – 24.114.43.194
@@ Line 30: / Line 30: @@
 While in the more advanced deep learning models, more hyperparameters will be involved. For examples, Artificial neural network (ANN) can include input layer, hidden layers and output layer connected by lines, and in each layer they can have certain number of nodes, which looks like a neuron network. Deep learning is one type of machine learning methods based on neural network, automatically extract features without manually prepare the features to be trained. If the output is incorrect, the network can re-adjust the weights of nodes to improve performance (back-propagation). Users can have many hyperparameters to re-tune the model, for examples:
-(1) the number of hidden layers;
+  * (1) the number of hidden layers;
-(2) the learning rate (which controls how the new parameters are set after back-propagation);
+  * (2) the learning rate (which controls how the new parameters are set after back-propagation);
-(3) how many instances to give the ANN at a time (batch size which is a number of samples processed before the model is updated. It must be more than or equal to one and less than or equal to the number of samples in the training dataset);
+  * (3) how many instances to give the ANN at a time (batch size which is a number of samples processed before the model is updated. It must be more than or equal to one and less than or equal to the number of samples in the training dataset);
-(4) how many times the NN should be trained on the entire input datasets (Epoch is defined as the total number of iterations for training the machine learning model with all the training data in one cycle. It is allowing the learning algorithm to run until the error from the model has been sufficiently minimized);
+  * (4) how many times the NN should be trained on the entire input datasets (Epoch is defined as the total number of iterations for training the machine learning model with all the training data in one cycle. It is allowing the learning algorithm to run until the error from the model has been sufficiently minimized);
-(5) how many epochs it must continue after the loss stopped from decreasing (patience). If you don't set patience value, the training will continue for all the epochs you set, even if your training results are not getting any better.
+  * (5) how many epochs it must continue after the loss stopped from decreasing (patience). If you don't set patience value, the training will continue for all the epochs you set, even if your training results are not getting any better.
-(6) early stopping is a form of regularization used to avoid overfitting when training a learner with an iterative method, such as gradient descent.
+  * (6) early stopping is a form of regularization used to avoid overfitting when training a learner with an iterative method, such as gradient descent.
 Sometimes, there are deep learning-specific parameters such as the models of convolutional neural network (CNN), recurrent neural network (RNN) and probabilistic neural network (PNN). CNN often used for image classification which utilize a convolution layer that preserves positional relationships between inputs (e.g. pixels in an image), thus capturing any dependencies among inputs. RNN discovers conditional dependencies of inputs by utilizing the output of previous inputs as features in classification, and can help model long range interactions. PNN estimates the probability distribution of each class, and the class with the highest posterior probability is assigned to each input.
@@ Line 43: / Line 43: @@
 Fortunately, the new developed platforms (e.g., DataCamp), and tools like tensor flow and keras making it easier to use deep learning algorithms which were built-in as python library. Although the barrier has been lower to access the door, it is still challenging to optimize / re-tune the model to acquire best predictions. Although deep leaning require larger amount of training  data  and computational power comparing to other typical machine learning methods (such as, random forests (RF), support vector machine (SVM) k-nearest neighbors (kNN)),  their accuracy evaluation methods are similar. For the classification model of deep learning or machine learning, Overall classification performance can be quantified by different metrics: TNR = true negative rate (also specificity), TPR = true positive rate (also sensitivity, recall), precision, accuracy, area under the receiver operating characteristic (AUROC) curve, PRC = precision-recall curve. For all of the above metrics, values closer to 1 indicate increasingly optimal performance. L is the positive class. TP = true positive, FP = false positive, TN = true negative, FN = false negative.
-{{:performace.png?600|}}
+{{:performace.png?400|}}
 For the input data set may be split into a Training Set (usually 75–90% of the input data) and a Test Set. When a Test Set cannot be made, cross validation (CV) may be sufficient to estimate the algorithm’s error in classification of test instances. Here, five-fold CV is shown. In each fold, 20% of the input training data is randomly chosen to be the Validation Set. The model is trained using the training data, and its performance on new instances is determined through the Validation Set. Once the benchmark dataset has been spited into test data and train data, they will be fitted in different models to evaluate the performance. Since the data points in the test set may not be representative of the model's ability to generalize to unseen data. To combat this dependence on what is essentially an arbitrary split, we use a technique called cross-validation.