The explosion of data is not to be underestimated. In fact, every time a new study is done the numbers are continually being revised upwards. Technology and internet solutions are moving at such a blistering pace and the consumerization of hardware is making it impossible to predict how much data we will actually be creating this decade.

In November 2018, IDC Corporate released a report entitled The Digitization of the World From Edge to Core. In the report, the market research company analyses the datasphere as it is and also predicts how it will look in the future. According to the report, IDC has specified three main locations in which where digitization is occurring and where digital content is created. These are the core, which refers to traditional and cloud datacenters; the edge, which refers to infrastructure such as cell towers and branch offices; and the endpoints, which refer to computers, smartphones and IoT devices.

All this data is collated within the Global Datasphere, which is currently undergoing massive expansion. In fact, IDC predicts that by the year 2025 the Global Datasphere will have expanded from 33 Zettabytes (ZB) in 2018 to a whopping 175 ZB. Writing for Network World, Andy Patrizio puts this number into perspective, pointing out that a zettabyte is the equivalent of a trillion gigabytes.

The challenge over the next decade will not only be to quantify the data accurately but how we are going to process it and where we will store it. With such a substantial increase in data creation, there are obvious concerns about where it will be stored. The report itself states that the demand for storage capacity is rapidly increasing amongst multiple consumers, enterprises, industries and governments. 

In the report, IDC predicts that “over 22ZB of storage capacity must ship across all media types from 2018 to 2025 to keep up with storage demands.” It also says that, over this time period, around 59% of that capacity will be supplied from the Hard Disk Drive (HDD) industry and 26% of it from flash technology. It also predicts that “49% of the world’s stored data will reside in public cloud environments.”

There is also the matter of human interaction with data. IDC speculates that, at the time of the report’s publication, over 5 billion people had been interacting with data on a daily basis, and that this number will grow to 6 billion by 2025. This amounts to 75% of the world’s population. Added to that, 2025 will, according to IDC, see every connected person experience at least one data interaction every 18 seconds. The constancy of these interactions will mainly be due to the numerous devices connected around the world thanks to the Internet of Things (IoT). These devices are expected to create more than 90 ZB of data in 2025.

Data is being generated at a rapid rate, perhaps at a rate that’s even faster than the time we have to process and store it. It’s possible that this might ultimately be one of the biggest challenges we face.