Data Related Guides
Big Data
| |
Small data is data that can be processed from a single machine.Big data is data that is too large to represent on a single machine such that distributed computing is required to process and store it.
The DifferenceBig data is heavily associated with cloud computing and platforms that can store and process data as a single collection that is physically distributed across a large number of machines. Cloud computing allows data storage and processing to be scaled without a technical limit with the only constraint being the cost of data centers, machines and electricity.Data that is not considered big data is typically called regular data or small data. Small data can be stored and processed on a single machine. As such, the exact limit of what is considered small data changes with time. At present, small data is usually less than 10 terabytes. This isn't actually small, depending on how you look at it. Small data can easily include millions of individual database records or files.
ExampleA common example of regular data are small websites. For example, a stamp collector website has 177,000 members. The site stores all registration information, forum posts and media such as images on a single machine that is backed up nightly. The site uses traditional database technologies.A social media platform handles 500 million posts per day representing several petabytes of data. This requires a specialized cloud platform that can process posts across thousands of geographically distributed machines. |
| Small Data | Big Data | | Definition | Data that can be stored and processed on a single machine. | Data that requires distributed computing for storage and processing. | | Maximize Size | 10 terabytes* | Limited only by capital and electricity, no technical limit. | | Example | A customer database of a single firm that includes 400,000 records. | A cloud platform that manages the customer databases of 4,000 firms. |
Notes* approximate effective limit as of this post. It is possible to go beyond this limit on a single machine but this is currently uncommon.
Data
This is the complete list of articles we have written about data.
If you enjoyed this page, please consider bookmarking Simplicable.
© 2010-2023 Simplicable. All Rights Reserved. Reproduction of materials found on this site, in any form, without explicit permission is prohibited.
View credits & copyrights or citation information for this page.
|
