cgeb2001's DokuWiki!

User Tools

Site Tools

Trace: • bioinformatics_essay36
bioinformatics_essay36

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
bioinformatics_essay36 [2023/08/15 19:40] – 72.143.225.209bioinformatics_essay36 [2023/08/16 12:52] (current) – 24.114.43.194
Line 1: Line 1:
-**A hyperparameter tuning game– Deep Learning**+**Deep Learning - A hyperparameter tuning game **
  
  {{::tuning_parameters.png?600|}}  {{::tuning_parameters.png?600|}}
Line 5: Line 5:
 Imagine one day you could look like alchemist to play magic or player in ball balance game to adventure. Imagine one day you could look like alchemist to play magic or player in ball balance game to adventure.
  
-To understand the main point of this essay, it will be helpful to refer to another one [[Bioinformatics_essay25| Machine learning made easy]]. As for machine learning method, it usually includes six steps whether or not it is regression or classification model:+To better understand the main point of this essay, it will be helpful to brush up some basics from another one [[Bioinformatics_essay25| Machine learning made easy]]. As for machine learning method, it usually includes six steps whether or not it is regression or classification model:
   - Loading the data   - Loading the data
   - Create your model   - Create your model
Line 24: Line 24:
       * finally you will want to use your model to make predictions       * finally you will want to use your model to make predictions
  
-No matter machine learning or more advanced deep learning algorithms, they all include a necessary step to re-tune the hyperparameters of the models being used. Hyperparameters are also known as human-tunable parameters but can be different when referring to various machine learning and artificial intelligence strategies. For examples, (1) in the Random Forest (RF) classifier, each tree is only allowed to consider a randomly chosen subset of features at each decision split. Users can specify the number of decision trees in the ensemble, the number of features to consider at each split, and the minimum number of instances per region; (2) In the Support Vector Machine (SVM) classifier, the binary classification is determined by the widest possible boundary between classes. Users can tune the parameter C, which defines the maximum number of misclassified instances allowed when maximizing the margin size; (3) In the K nearest neighbour (kNN) classifier, it predicts the label of a given instance based upon the majority label of its k nearest neighbors. Users can tune the value of k to acquire the best results.+No matter machine learning or more advanced deep learning algorithms, they all include a necessary step to re-tune the hyperparameters of the models being used. Hyperparameters are also known as human-tunable parameters but can be different when referring to various machine learning and artificial intelligence strategies. For examples:  
 +  - In the Random Forest (RF) classifier, each tree is only allowed to consider a randomly chosen subset of features at each decision split. Users can specify the number of decision trees in the ensemble, the number of features to consider at each split, and the minimum number of instances per region;  
 +  - In the Support Vector Machine (SVM) classifier, the binary classification is determined by the widest possible boundary between classes. Users can tune the parameter C, which defines the maximum number of misclassified instances allowed when maximizing the margin size; 
 +  - In the K nearest neighbour (kNN) classifier, it predicts the label of a given instance based upon the majority label of its k nearest neighbors. Users can tune the value of k to acquire the best results.
  
-While in the more advanced deep learning models, more hyperparameters will be involved. For examples, Artificial neural network (ANN) can include input layers, hidden layers and output layers connected by lines, and in each layer they can have certain number of nodes, which looks like a neuron network. Deep learning is one type of machine learning methods based on neural network, automatically extract features without manually prepare the features to be trained. If the output is incorrect, the network can re-adjust the weights of nodes to improve performance (back-propagation). Users can have many hyperparameters to re-tune the model, e.g., (1) the number of hidden layers; (2) the learning rate (which controls how the new parameters are set after back-propagation); (3) how many instances to give the ANN at a time (batch size which is a number of samples processed before the model is updated. It must be more than or equal to one and less than or equal to the number of samples in the training dataset); (4) how many times the NN should be trained on the entire input datasets (Epoch is defined as the total number of iterations for training the machine learning model with all the training data in one cycle. It is allowing the learning algorithm to run until the error from the model has been sufficiently minimized); (5) how many epochs it must continue after the loss stopped from decreasing (patience). If you don't set patience value, the training will continue for all the epochs you set, even if your training results are not getting any better. (6) early stopping is a form of regularization used to avoid overfitting when training a learner with an iterative method, such as gradient descent.+While in the more advanced deep learning models, more hyperparameters will be involved. For examples, Artificial neural network (ANN) can include input layer, hidden layers and output layer connected by lines, and in each layer they can have certain number of nodes, which looks like a neuron network. Deep learning is one type of machine learning methods based on neural network, automatically extract features without manually prepare the features to be trained. If the output is incorrect, the network can re-adjust the weights of nodes to improve performance (back-propagation). Users can have many hyperparameters to re-tune the model, for examples:  
 +  * (1) the number of hidden layers;  
 +  * (2) the learning rate (which controls how the new parameters are set after back-propagation);  
 +  * (3) how many instances to give the ANN at a time (batch size which is a number of samples processed before the model is updated. It must be more than or equal to one and less than or equal to the number of samples in the training dataset);  
 +  * (4) how many times the NN should be trained on the entire input datasets (Epoch is defined as the total number of iterations for training the machine learning model with all the training data in one cycle. It is allowing the learning algorithm to run until the error from the model has been sufficiently minimized);  
 +  * (5) how many epochs it must continue after the loss stopped from decreasing (patience). If you don't set patience value, the training will continue for all the epochs you set, even if your training results are not getting any better.  
 +  * (6) early stopping is a form of regularization used to avoid overfitting when training a learner with an iterative method, such as gradient descent.
  
 Sometimes, there are deep learning-specific parameters such as the models of convolutional neural network (CNN), recurrent neural network (RNN) and probabilistic neural network (PNN). CNN often used for image classification which utilize a convolution layer that preserves positional relationships between inputs (e.g. pixels in an image), thus capturing any dependencies among inputs. RNN discovers conditional dependencies of inputs by utilizing the output of previous inputs as features in classification, and can help model long range interactions. PNN estimates the probability distribution of each class, and the class with the highest posterior probability is assigned to each input. Sometimes, there are deep learning-specific parameters such as the models of convolutional neural network (CNN), recurrent neural network (RNN) and probabilistic neural network (PNN). CNN often used for image classification which utilize a convolution layer that preserves positional relationships between inputs (e.g. pixels in an image), thus capturing any dependencies among inputs. RNN discovers conditional dependencies of inputs by utilizing the output of previous inputs as features in classification, and can help model long range interactions. PNN estimates the probability distribution of each class, and the class with the highest posterior probability is assigned to each input.
Line 34: Line 43:
 Fortunately, the new developed platforms (e.g., DataCamp), and tools like tensor flow and keras making it easier to use deep learning algorithms which were built-in as python library. Although the barrier has been lower to access the door, it is still challenging to optimize / re-tune the model to acquire best predictions. Although deep leaning require larger amount of training  data  and computational power comparing to other typical machine learning methods (such as, random forests (RF), support vector machine (SVM) k-nearest neighbors (kNN)),  their accuracy evaluation methods are similar. For the classification model of deep learning or machine learning, Overall classification performance can be quantified by different metrics: TNR = true negative rate (also specificity), TPR = true positive rate (also sensitivity, recall), precision, accuracy, area under the receiver operating characteristic (AUROC) curve, PRC = precision-recall curve. For all of the above metrics, values closer to 1 indicate increasingly optimal performance. L is the positive class. TP = true positive, FP = false positive, TN = true negative, FN = false negative.  Fortunately, the new developed platforms (e.g., DataCamp), and tools like tensor flow and keras making it easier to use deep learning algorithms which were built-in as python library. Although the barrier has been lower to access the door, it is still challenging to optimize / re-tune the model to acquire best predictions. Although deep leaning require larger amount of training  data  and computational power comparing to other typical machine learning methods (such as, random forests (RF), support vector machine (SVM) k-nearest neighbors (kNN)),  their accuracy evaluation methods are similar. For the classification model of deep learning or machine learning, Overall classification performance can be quantified by different metrics: TNR = true negative rate (also specificity), TPR = true positive rate (also sensitivity, recall), precision, accuracy, area under the receiver operating characteristic (AUROC) curve, PRC = precision-recall curve. For all of the above metrics, values closer to 1 indicate increasingly optimal performance. L is the positive class. TP = true positive, FP = false positive, TN = true negative, FN = false negative. 
  
-{{:performace.png?600|}}+{{:performace.png?400|}}
  
 For the input data set may be split into a Training Set (usually 75–90% of the input data) and a Test Set. When a Test Set cannot be made, cross validation (CV) may be sufficient to estimate the algorithm’s error in classification of test instances. Here, five-fold CV is shown. In each fold, 20% of the input training data is randomly chosen to be the Validation Set. The model is trained using the training data, and its performance on new instances is determined through the Validation Set. Once the benchmark dataset has been spited into test data and train data, they will be fitted in different models to evaluate the performance. Since the data points in the test set may not be representative of the model's ability to generalize to unseen data. To combat this dependence on what is essentially an arbitrary split, we use a technique called cross-validation. For the input data set may be split into a Training Set (usually 75–90% of the input data) and a Test Set. When a Test Set cannot be made, cross validation (CV) may be sufficient to estimate the algorithm’s error in classification of test instances. Here, five-fold CV is shown. In each fold, 20% of the input training data is randomly chosen to be the Validation Set. The model is trained using the training data, and its performance on new instances is determined through the Validation Set. Once the benchmark dataset has been spited into test data and train data, they will be fitted in different models to evaluate the performance. Since the data points in the test set may not be representative of the model's ability to generalize to unseen data. To combat this dependence on what is essentially an arbitrary split, we use a technique called cross-validation.
bioinformatics_essay36.1692139221.txt.gz · Last modified: by 72.143.225.209