JAWAHARLAL NEHRU TECHNOLOGICAL UNIVERSITY HYDERABAD B.Tech III MACHINE LEARNING Previous Question Paper (March - 2021 - R18)

Decision Tree Learning is a supervised learning method used for classification and regression. It represents data as a tree structure where internal nodes represent tests on attributes, branches represent the outcome of tests, and leaf nodes represent class labels. The algorithm recursively partitions the training data into subsets using a greedy approach, typically selecting the attribute that maximizes Information Gain or minimizes Gini Impurity.

For example, consider a dataset to predict if a person will play tennis based on 'Outlook' (Sunny, Overcast, Rain) and 'Humidity' (High, Normal). The algorithm calculates the entropy of the target variable and then the information gain for each attribute. If 'Outlook' provides the highest information gain, it becomes the root node. The data is then split into subsets for each branch (Sunny, Overcast, Rain), and the process repeats for each subset until a stopping criterion is met, such as all samples in a node belonging to the same class.

Decision tree learning faces several key challenges:

• Overfitting: Trees often grow too complex, capturing noise in the training data rather than the underlying pattern. This is typically mitigated through pruning techniques.
• Instability: Small variations in the input data can lead to significantly different tree structures, making the model sensitive to noise.
• Greedy Nature: Because the algorithm makes locally optimal choices at each node, it does not guarantee a globally optimal tree.
• Bias towards attributes with many levels: Attributes with a large number of distinct values can lead to higher information gain, potentially causing the tree to favor these features even if they are not the most predictive.
• Complexity with continuous variables: Handling continuous data requires sorting and evaluating potential split points, which increases computational overhead.