Informatics Practices: Class 11 — Chapter 5: Understanding Data

5.1 Introduction to Data

In modern society, decisions are heavily based on data and information. Whether choosing a college based on placement records, fees, and faculty qualifications, or governments conducting a census to systematically record population details, data serves as the foundation for analysis and policy formulation.

"Data is not information, Information is not knowledge, Knowledge is not understanding, Understanding is not wisdom."


Socio-Technical Analysis & Core Systemic Impact: This quote by Gary Schubert highlights the cognitive hierarchy in data science. Raw data must undergo structured processing to become meaningful information, which is then contextualized into knowledge, synthesized into understanding, and ultimately applied as wisdom for decision-making.

Technical Profile: Data vs. Datum

  • Architecture Layer / Tier: Conceptual Data Layer
  • Operational Mechanics: Data is a plural term representing a collection of characters, numbers, and other symbols representing values of situations or variables. The singular form of the word is datum .
  • System Impact & Execution Outputs: Storing data electronically using computers enables faster, automated processing compared to manual, error-prone human methods.
Common Examples of Data
  • Personal details: Name , age , gender , contact details .
  • Transaction logs: Banking, ticketing, and shopping transactions (online or offline).
  • Multimedia: Images, graphics, animations, audio, and video files.
  • Web content: Documents, web pages, online posts, comments, and messages.
  • Sensor outputs: Signals generated by IoT devices and physical sensors.
  • Satellite feeds: Meteorological data, communication signals, and earth observation data.
5.1.1 Importance of Data

Data processing reveals hidden patterns and possibilities that are otherwise invisible to humans. Key real-world applications include:

  • Banking: Real-time updates of account balances during ATM withdrawals.
  • Meteorology: Continuous satellite monitoring to predict cyclones, heavy rainfall, and natural disasters.
  • Business Analytics: Monitoring market behavior, customer feedback, and demand-supply relationships.
  • Dynamic Pricing: Algorithms used by airlines, railways, and cab-booking apps to adjust prices dynamically based on real-time demand.
  • Elections: Electronic Voting Machines (EVMs) recording and accumulating votes for rapid, secure counting.
5.1.2 Types of Data

Data can be broadly classified into two categories based on its format and organization:

(A) Structured Data

Data that is organized and recorded in a well-defined format is called structured data . It is typically stored in a tabular format consisting of rows and columns, where each column represents an attribute (parameter/characteristic) and each row represents an individual observation.

Table 5.1: Structured data about kitchen items in a shop

ModelNo ProductName Unit Price Discount(%) Items_in_Inventory
ABC1 Water bottle 126 8 13
ABC2 Melamine Plates 320 5 45
ABC3 Dinner Set 4200 10 8
GH67 Jug 80 0 10
GH78 Table Spoon 120 5 14
GH81 Bucket 190 12 6
NK2 Kitchen Towel 25 0 32

Table 5.2: Attributes maintained for different activities

Entity/Activities Data Fields/Parameters/Attributes
Books at a shop BookTitle , Author , Price , YearofPublication
Depositing fees in a school StudentName , Class , RollNo , FeesAmount , DepositDate
Amount withdrawal from ATM AccHolderName , AccountNo , TypeofAcc , DateofWithdrawal , AmountWithdrawn , ATMid , TimeOfWithdrawal
(B) Unstructured Data

Data that does not conform to a traditional row-and-column structure is called unstructured data . Examples include newspapers (variable layouts, mixed \text and images), emails, web pages, audio/video files, and social media posts.

Technical Profile: Metadata

  • Architecture Layer / Tier: Data Description Layer
  • Operational Mechanics: Metadata is defined as "data about data". It provides structural context to unstructured or semi-structured files.
  • System Impact & Execution Outputs: For an email, metadata includes fields like subject , recipient , and attachment details . For an image file, metadata includes image size (in KB/MB), image type (JPEG, PNG), and resolution .
5.2 Data Collection

Data collection involves identifying existing data or gathering new data from appropriate sources to prepare it for processing and storage.

Consider three common scenarios for collecting sales data in a grocery store:

  1. Manual Records: Data is recorded in a physical diary or register. This must be manually digitized into a spreadsheet or database.
  2. Semi-Structured Files: Data is already available in a digital format, such as a CSV (Comma Separated Values) file.
  3. No Prior Records: No data has been recorded. A custom software application (e.g., built using Python and MySQL ) must be developed to capture and store transactions in real-time.
5.3 Data Storage

Data storage is the process of saving processed or raw data on physical or digital storage devices for future retrieval and reuse. With the exponential growth of data, storage has become a critical challenge, mitigated by the falling cost of digital storage media.

5.4 Data Processing

Raw data (numbers, \text, images) cannot be directly used for decision-making. It must undergo a systematic transformation process to become meaningful information.

Technical Profile: The Data Process Cycle

  • Input Phase: Data Collection, Data Preparation, and Data Entry.
  • Processing Phase: Storage, Retrieval, Classification, and Updating.
  • Output Phase: Generation of Reports, Results, and actionable Information.
Real-World Data Processing Workflows
Problem Statement Inputs Collected Processing Steps Generated Output
Online Student Registration & Admit Card Generation Student details (name, address, qualification, marks, photo, signature, online fee payment details). Verify correctness of data, check eligibility criteria, validate fee payment, verify photo/signature uploads, generate roll number, and append to the eligible applicant database. Examination Admit Card specifying roll number, center address, date, and time of test.
ATM Cash Withdrawal ATM PIN, account type, account number, card number, ATM location, date/time, and withdrawal amount. Validate PIN, check account balance, deduct requested amount from the account, count physical currency notes, and initiate receipt printing. Dispensed currency notes and a printed transaction slip.
Train Ticket Booking Journey start/end stations, date, number of tickets, class of travel, berth preference, passenger names/ages, contact details, and payment details. Verify login credentials, check berth availability, process payment, deduct booked seats from inventory, allocate PNR and berth numbers (or generate a waiting list number). Confirmed train ticket with berth/coach details or a waitlisted ticket.
5.5 Statistical Techniques for Data Processing

Statistical techniques help summarize tabular data to reveal its underlying characteristics. These are broadly divided into measures of central tendency and measures of variability.

5.5.1 Measures of Central Tendency

A measure of central tendency provides a single representative value that summarizes a dataset.

(A) Mean

The Mean (or average) is the sum of all numeric values in an attribute divided by the total number of values.

Mathematical Definition:

Given $n$ values $x_1, x_2, x_3, \dots, x_n$, the mean $\bar{x}$ is computed as:

$$\bar{x} = \frac{\sum_{i=1}^{n} x_i}{n}$$

Example 5.1:

Height (in cm) of 9 students: [90, 102, 110, 115, 85, 90, 100, 110, 110]

$$\bar{x} = \frac{90 + 102 + 110 + 115 + 85 + 90 + 100 + 110 + 110}{9} = \frac{912}{9} = 101.33\text{ cm}$$

Outlier Warning: An outlier is an exceptionally large or small value compared to the rest of the dataset. The mean is highly sensitive to outliers, which can distort the average. Outliers should be identified and handled before calculating the mean.

(B) Median

The Median is the middle value of a dataset when the values are sorted in ascending or descending order. It divides the dataset into two equal halves.

  • If $n$ is odd , the median is the value at the middle position: $\frac{n+1}{2}$.
  • If $n$ is even , the median is the average of the two middle values.

Example 5.2:

Sorting the student heights from Example 5.1: [85, 90, 90, 100, 102, 110, 110, 110, 115]

Since $n = 9$ (odd), the median is the value at position 5, which is 102 cm .

(C) Mode

The Mode is the value that appears most frequently in a dataset. It can be calculated for both numeric and non-numeric data.

  • A dataset has no mode if all values occur only once.
  • A dataset can have multiple modes (multimodal) if multiple values share the highest frequency.

Example 5.3:

In the student heights dataset, the value 110 occurs 3 \times (more than any other value). Thus, the mode is 110 cm .

5.5.2 Measures of Variability

Measures of variability (or dispersion) describe how spread out or diverse the data values are around the mean.

(A) Range

The Range is the difference between the maximum and minimum values in a dataset. It is highly sensitive to outliers.

$$\text{Range} = \text{Maximum} - \text{Minimum}$$

Example 5.4:

For the student heights dataset, the maximum height is 115 cm and the minimum is 85 cm.

$$\text{Range} = 115 - 85 = 30\text{ cm}$$

(B) Standard Deviation

The Standard Deviation ($\sigma$) measures the spread of data by calculating the positive square root of the average of the squared differences of each value from the mean. Unlike range, it considers all data points.

Mathematical Formula:

$$\sigma = \sqrt{\frac{\sum_{i=1}^{n} (x_i - \bar{x})^2}{n}}$$

Example 5.5 & Table 5.3: Standard deviation calculation for 9 students ($\bar{x} = 101.33$)

Height ($x$) in cm $x - \bar{x}$ $(x - \bar{x})^2$
90 -11.33 128.37
102 0.67 0.36
110 8.67 75.17
115 13.67 186.87
85 -16.33 266.67
90 -11.33 128.37
100 -1.33 1.77
110 8.67 75.17
110 8.67 75.17
$n = 9$
$\bar{x} = 101.33$
$\sum(x - \bar{x}) = 0.03$ $\sum(x - \bar{x})^2 = 938.00$

Calculation Steps:

  1. Calculate the variance: $$\text{Variance} = \frac{\sum(x - \bar{x})^2}{n} = \frac{938.00}{9} = 104.22$$
  2. Calculate the standard deviation: $$\sigma = \sqrt{104.22} \approx 10.2\text{ cm}$$
Selecting Suitable Statistical Techniques
Problem Statement Suitable Statistical Method
The management of a company wants to know about disparity in salaries of all employees. Standard Deviation / Range
Teacher wants to know about the average performance of the whole class in a test. Mean
Compare height of residents of two cities. Mean (for average height) and Standard Deviation (for variation)
Find the dominant value from a set of values. Mode
Compare income of residents of two cities. Median (less sensitive to extreme income outliers)
Find the popular color for car after surveying the car owners of a small city. Mode