Big data, its infrastructure and data science techniques

The five Vs

Big data is data that is too large, fast or varied to be stored and processed using traditional methods (a single server and relational database).

• Volume: very large quantities (terabytes to petabytes and beyond).
• Velocity: data arrives very fast and often must be processed in real time, for example sensor readings or card payments.
• Variety: many types and formats: structured (tables), semi-structured (JSON, logs) and unstructured (text, images, video).
• Veracity: how accurate and trustworthy the data is; big data is often incomplete, inconsistent or biased.
• Value: the useful insight that can be gained; data is only worth collecting if it leads to better decisions.

Infrastructure

A big data system has stages:

• Collection: data gathered from sensors, websites, apps, transactions and social media.
• Storage: distributed across many servers (often in the cloud), as no single machine can hold it; non-relational stores handle varied data.
• Processing: work is split across many machines in parallel (distributed processing) to cope with volume and velocity.
• Modelling and analysis: statistical models and machine learning find patterns and make predictions.
• Networking: high-bandwidth connections link the servers and move data between stages.
• Security: access control, encryption and compliance with data protection law to protect personal data.

Data science tools and techniques

• Collection: surveys, sensors, logs, APIs.
• Storage: databases, data warehouses and cloud storage.
• Web scraping: software that automatically extracts data from web pages. Fast and cheap; but may break site terms of use or copyright, and the data may be unreliable.
• Cleaning: removing duplicates, fixing errors, handling missing values, making formats consistent. Essential, because analysis of dirty data gives misleading results.
• Analysis: statistics and machine learning to find trends, correlations and predictions.
• Visualisation: charts, maps and dashboards that make findings easy to understand.

Benefits of big data: better decisions, personalised services, fraud detection, medical research. Drawbacks: cost of infrastructure, privacy and surveillance concerns, security risks, and biased or poor-quality data leading to wrong conclusions.

Worked example

A city fits sensors to buses that report location every 5 seconds. Identify which of the five Vs are shown and how the data could provide value.

1. Velocity: readings arrive continuously and must be processed in real time.
2. Volume: thousands of buses x 17 280 readings per day each.
3. Value: live arrival times for passengers and better timetables.

Answer: Velocity and volume, giving value through live tracking and planning.

Check yourself

1. Which V of big data refers to how accurate and trustworthy the data is?
   • Velocity
   • Variety
   • Veracity
   • Volume
   Show answer

   Veracity. Veracity = truthfulness.

2. A company stores emails, videos, spreadsheets and sensor logs together. Which V does this show?
   • Variety
   • Velocity
   • Veracity
   • Value
   Show answer

   Variety. Many different formats.

3. Why must data be cleaned before analysis?
   • Cleaning makes the data set larger
   • Cleaning encrypts the data
   • Errors, duplicates and missing values would give misleading results
   • Analysis software cannot read numbers
   Show answer

   Errors, duplicates and missing values would give misleading results. Garbage in, garbage out.

4. What is web scraping?
   • Deleting old web pages from a server
   • Using software to extract data automatically from web pages
   • Encrypting web traffic
   • Designing the layout of a website
   Show answer

   Using software to extract data automatically from web pages. It collects data at scale from websites.

5. A sensor sends one reading every 5 seconds. How many readings does it send in one day?
   Show answer

   17280. 24 x 60 x 60 = 86 400 seconds; / 5 = 17 280 readings.

6. Explain why big data is stored and processed across many servers.
   Show answer

   The volume is too large for one machine to store and the velocity too high for one machine to process, so the work is split and done in parallel across many servers.. Link volume to storage and velocity to parallel processing.

7. Which is a drawback of collecting big data about people?
   • It always produces correct conclusions
   • It needs no storage
   • It cannot be analysed by computers
   • It raises privacy concerns and must be protected against breaches
   Show answer

   It raises privacy concerns and must be protected against breaches. Personal data can be misused or leaked.

Practise with instant feedback Print this lesson

Sources

• Pearson Edexcel International Advanced Subsidiary/Advanced Level in Computer Science specification (2026)

Last updated 8 November 2026. Spotted a mistake? Tell us.