JNTU B.Tech IV Year DATA ANALYTICS Previous Question Paper (December - 2020 - R16)

An outlier is a data point that differs significantly from other observations in a dataset, potentially indicating measurement error, experimental variability, or novel phenomena. Detecting outliers is crucial for ensuring the accuracy of statistical models and data analysis.

The most common method for detecting outliers in numerical data is the Interquartile Range (IQR) method. The steps for this method are:

• Calculate the first quartile (Q1), which is the 25th percentile, and the third quartile (Q3), which is the 75th percentile of the data.
• Determine the Interquartile Range (IQR) using the formula:
  IQR = Q3 - Q1
• Define the lower bound and upper bound for acceptable data points:
  Lower Bound = Q1 - 1.5 × IQR
  Upper Bound = Q3 + 1.5 × IQR
• Any data point falling below the lower bound or above the upper bound is classified as an outlier.

Example: Consider the dataset: {10, 12, 12, 13, 14, 15, 16, 17, 18, 50}.

1. Sort the data: {10, 12, 12, 13, 14, 15, 16, 17, 18, 50}.
2. Identify Q1 and Q3: With 10 observations, Q1 (the median of the lower half) is 12, and Q3 (the median of the upper half) is 17.
3. Calculate IQR:
   IQR = 17 - 12 = 5
4. Calculate bounds:
   Lower Bound = 12 - (1.5 × 5) = 4.5
   Upper Bound = 17 + (1.5 × 5) = 24.5
5. Conclusion: Since 50 is greater than the upper bound of 24.5, it is identified as an outlier.

Other methods include the Z-score method, which identifies outliers as points with a Z-score greater than 3 or less than -3 (assuming a normal distribution), and Visualization techniques like Box Plots, where outliers are represented as individual points beyond the whiskers.

Design data architecture is a framework that defines how data is collected, stored, processed, and consumed within an organization. It ensures that data is accessible, secure, and reliable for analytical purposes. The primary elements of data architecture are described below:

1. Data Sources: These are the origins of raw data, including transactional databases (RDBMS), cloud applications, IoT devices, social media feeds, and external APIs.

2. Data Ingestion Layer: This layer is responsible for moving data from sources to the storage environment. It uses ETL (Extract, Transform, Load) or ELT processes to handle batch processing and real-time streaming.

3. Data Storage Layer: This is the central repository where data resides. It typically includes a Data Warehouse for structured data, a Data Lake for raw/unstructured data, and sometimes a Data Lakehouse to combine the benefits of both.

4. Data Processing Layer: This layer transforms raw data into meaningful insights. It involves data cleansing, aggregation, and complex transformations using tools like Apache Spark, Hadoop, or cloud-native processing engines.

5. Data Access and Consumption Layer: This is the interface for end-users and applications. It includes Business Intelligence (BI) tools, data visualization dashboards, and machine learning models that consume the processed data.

6. Data Governance and Security: A cross-cutting element that ensures data quality, compliance with regulations (like GDPR), and security through encryption, access control, and metadata management.

Diagram Representation:

The architecture is supported by a foundation of Data Governance, which manages the metadata, security, and quality standards across all these layers to ensure the integrity of the entire data pipeline. Effective Data Analytics lifecycle. Each element must be scalable to handle the increasing volume, velocity, and variety of modern data.