JNTU B.Tech III MACHINE LEARNING Previous Question Paper (January/February - 2023 - R18)

Machine learning systems encounter several fundamental challenges that can hinder performance, reliability, and scalability. The primary issues include:

• Data Quality and Quantity: Models are highly sensitive to the quality of training data. Issues such as noisy data, missing values, outliers, and insufficient sample sizes often lead to poor generalization.
• Overfitting and Underfitting: Overfitting occurs when a model learns the training data too well, including its noise, resulting in poor performance on unseen data. Conversely, underfitting happens when a model is too simple to capture the underlying patterns of the data.
• Bias and Variance Trade-off: Achieving a balance between bias (error from erroneous assumptions) and variance (sensitivity to small fluctuations in the training set) is a core difficulty in model optimization.
• Computational Complexity: Many algorithms, particularly deep learning models, require significant memory and processing power, making them difficult to deploy in resource-constrained environments.
• Interpretability: Often referred to as the "black box" problem, many complex models (like neural networks) provide high accuracy but lack transparency, making it difficult to understand the reasoning behind specific predictions.
• Data Privacy and Security: Protecting sensitive information used during training and ensuring models are resilient against adversarial attacks are critical ongoing concerns.

In decision tree learning, the hypothesis space consists of the set of all possible decision trees that can be constructed from a given set of attributes. The search process involves navigating this space to find a tree that best fits the training data while maintaining good generalization capabilities.

• Search Strategy: Decision tree algorithms, such as ID3 or C4.5, typically employ a greedy, top-down heuristic search. This approach starts with an empty tree and iteratively selects the best attribute to split the data based on metrics like Information Gain or Gini Impurity.
• Completeness: The hypothesis space is extremely large, as it includes every possible combination of attribute tests. Because the algorithm makes locally optimal choices at each node, it performs a hill-climbing search rather than an exhaustive search of the entire space.
• Bias: The search is guided by an inductive bias that prefers shorter, simpler trees over complex ones, consistent with Occam's Razor. This bias helps prevent overfitting and ensures the model remains interpretable.
• Backtracking: Most standard decision tree algorithms do not backtrack. Once an attribute is chosen for a node, the algorithm proceeds to partition the data and continues the process recursively, meaning it does not revisit previous decisions to explore alternative branches of the hypothesis space.