Decision Trees in Machine Learning
The best way to understand Decision Trees goes through Machine Learning. An interdisciplinary field of study and a subset of Artificial Intelligence, Machine Learning enables computers to learn from huge amounts of data, without the need to be programmed.
There are three types of Machine Learning Models:
Supervised Learning — This happens when the dataset has labels and features that area already predefined on which the machine learning models have been trained. Supervised Learning further has two sub-parts: Classification and Regression.
Unsupervised Learning — A dataset with predefined features but has no labels then Machine Learning algorithms must operate the data to provide labels. This is also done to decrease the data’s dimensionality. The Clustering and Principal Component Analysis (PCA) are the most common among several types of Unsupervised Learning Models.
Reinforcement Learning — This is the most advanced type of Machine Learning model where it learns from ‘experience’. There is hardly any need to define features and labels. The Reinforcement Learning model is provided with a ‘situation’ and is compensated or reprimanded according to the ‘outcome’. To maximize the rewards, the model learns to optimize the ‘situation’. Thus, improving the ‘outcome’ with growing ‘experience’.
Classification process helps to determine which category does the data point belongs to. In this process, a Supervised Learning Algorithm learns to interpret from the given dataset’s features and predict the class, category, or group the data point belongs to. A simple example of this can be handwritten numbers from zero to nine. The idea is to teach the machine to categorize the correct image to the correct number. Thus, the machine is ‘trained’ in a way that it would correctly classify the numbers as per the image.
Face Recognition in smartphones and for biometric security as well as Medical image classification to understand fractures, malignant tumor (cancer), heart ailments, etc are the real-world applications of classification Algorithms.
Regression falls under supervised learning as well. Instead of predicting the class of given data like Classification, Regression predicts the corresponding values of the dataset created on the “features” it meets.
Stock market predictions and Object detection algorithm used to locate a given object in an image or video are real-world examples of regression algorithms.
Decision Tree Building Blocks
Entropy — Entropy is a typically used concept in information. A measure of ‘purity’ in the random compilation of information.
Information Gain — The calculation of the amount of relevant information that is achieved from a random sample size with the knowledge of Entropy is called Information Gain.
In simple terms, a Decision Tree is a tree that is created on some decisions taken by the algorithm as per the dataset that it has been trained on. In this analysis, each branch of the tree represents a possible decision, reaction, or occurrence.
The Decision Tree starts with a single node and branches out in conceivable outcomes. These outcomes further add nodes that branch out to more possibilities. Thus, giving the method a tree-like silhouette. Decision Tree Learning is a predictive modeling approach that works on the principle of conditions. This flow-like-a-tree structure has proved to be efficient as it predicts the expected result quite correctly.
The main benefits of the Decision Tree method are that it is quite simple to understand. There is not much effort that is required in data preparation. Its non-linear factor does not affect presentation. Decision Trees solve many business problems. Its applications can be found in different fields like Management, Engineering, Medicine and so on. Fundamentally, if you have data and you need to make decisions amongst uncertain conditions, the Decision Tree method is the way to go.
Along with Machine Learning, is used in data mining, and statistics. Decision analysis uses decision trees to represent decisions and decision making visually and explicitly.