JAWAHARLAL NEHRU TECHNOLOGICAL UNIVERSITY HYDERABAD MCA IV Semester BIGDATA ANALYTICS Previous Question Paper (September/October - 2022 - R20)

Data is categorized based on its organization and accessibility for processing. Structured data is highly organized, typically residing in relational databases (RDBMS) with fixed schemas. It is easily searchable, managed by SQL, and follows a strict tabular format (rows and columns). Examples include transaction records and inventory databases.

Unstructured data lacks a predefined data model or organizational structure. It accounts for the majority of data generated today and is difficult to analyze using traditional tools. Examples include emails, social media posts, videos, audio files, and images.

Semi-structured data sits between the two; it does not reside in a relational database but contains tags or markers to enforce a hierarchy and separate semantic elements. It is self-describing and flexible, allowing for evolving schemas. Common formats include JSON, XML, and YAML.

The primary contrast lies in schema enforcement: structured data requires schema-on-write, semi-structured data often uses schema-on-read, and unstructured data lacks a schema entirely, requiring advanced techniques like natural language processing or computer vision for extraction.

Big data presents significant technical hurdles due to its volume, velocity, and variety. Key challenges include:

• Storage and Scalability: Traditional storage systems cannot handle the petabyte-scale growth of data. Organizations must move to distributed file systems like HDFS or cloud-based object storage that allow horizontal scaling.
• Data Processing Speed: The velocity of incoming data, particularly from IoT sensors or real-time streams, requires high-throughput processing frameworks like Apache Spark or Flink to derive insights without significant latency.
• Data Quality and Integration: Combining data from disparate sources (structured, semi-structured, and unstructured) leads to noise, missing values, and inconsistencies. Cleaning and normalizing this data is computationally expensive and complex.
• Security and Privacy: Managing access control and ensuring compliance with data protection regulations becomes difficult when data is distributed across multiple nodes and geographic locations.
• Skill Gap and Tooling: Big data requires specialized expertise in distributed computing, machine learning, and data engineering, alongside the maintenance of complex ecosystems like the Hadoop stack.