The Difference
Big data is heavily associated with cloud computing and platforms that can store and process data as a single collection that is physically distributed across a large number of machines. Cloud computing allows data storage and processing to be scaled without a technical limit with the only constraint being the cost of data centers, machines and electricity.Data that is not considered big data is typically called regular data or small data. Small data can be stored and processed on a single machine. As such, the exact limit of what is considered small data changes with time. At present, small data is usually less than 10 terabytes. This isn't actually small, depending on how you look at it. Small data can easily include millions of individual database records or files.Example
A common example of regular data are small websites. For example, a stamp collector website has 177,000 members. The site stores all registration information, forum posts and media such as images on a single machine that is backed up nightly. The site uses traditional database technologies.A social media platform handles 500 million posts per day representing several petabytes of data. This requires a specialized cloud platform that can process posts across thousands of geographically distributed machines.| Small Data vs Big Data | ||
Small Data | Big Data | |
Definition | Data that can be stored and processed on a single machine. | Data that requires distributed computing for storage and processing. |
Maximize Size | 10 terabytes* | Limited only by capital and electricity, no technical limit. |
Example | A customer database of a single firm that includes 400,000 records. | A cloud platform that manages the customer databases of 4,000 firms. |
