A Decade in Data Storage: Looking Back at a Few of the Most Consequential Industry Developments

Reading time for this article .

The pace of technological innovation has continued to accelerate in the last decade, and in the data storage industry it has been no different.  In the 24 years that I’ve been in this industry I’ve witnessed it firsthand. As CTO at Spectra Logic, keeping my finger on the pulse of market trends and technology advancements has been key to accelerating Spectra’s solution roadmap and executing on the company’s technology strategy. Of the many technology advancements and industry developments that have marked the decade, a few must be noted due to their disproportionate impact on the future of data storage.

The Rise of Ransomware
From the early days of computer hacking, threat actors have always attempted to steal data from an organization or business – which is one of the primary purposes of computer viruses. Over the last 10 years, the goal has changed from stealing data to kidnapping the data. Today, cybercriminals use ransomware to hold an organization’s data hostage with the threat of destroying the data or selling it on the dark web if their monetary demands are not met in a timely fashion.

The rise of ransomware puts a premium on data and data protection. Over the last decade, as threat actors have become more sophisticated, they have realized that money can be made by electronically infiltrating an organization’s IT department and encrypting data that might include trade secrets, IP, customer lists, etc. while threatening to sell or destroy that critical data unless a ransom is paid – usually in bitcoin to prevent tracking by authorities. The larger the company the more valuable the haul in many cases. Because an organization’s data is most important to that entity, they will likely pay more to get it back than anyone on the black market.

IT workflows and systems have become faster and more resilient – enabling organizations to improve their IT infrastructures and better serve their internal and external customers. These complex networks, however, also leave organizations more vulnerable to the encryption of ransomware as, once the systems on the network are attacked, they often are encrypted at the speed of disk. Only those devices that store data out of the network stream, such as tape, are safe from ransomware’s impact.

The Old Mantra that ‘Tape is Dead’ is Dead
For the last decade, just as in decades before that, the industry has been pronouncing the death of tape storage. However, today, tape is back into a growth mode according to most analysts assessing the market. Tape, with recent R&D innovations and deployments now sits poised for another 10-plus years of success. Unlike the nearest competitor, the spinning archive disk, tape’s R&D hurdles are well understood, the physics are known and don’t need to be invented. Even most of the cloud storage providers know that tape has to be a part of the ecosystem from a cost and reliability standpoint. As cloud vendors develop cold tiers of storage at a fraction of a cent per month per gigabyte, each forcing the other to a lower and lower price, tape, with its low total cost of ownership, high reliability, scalable capacity and low power and cooling costs, is the logical solution. And tape’s traditional usage for archive in industries such as oil and gas, M&E, government, scientific research, and more, is seeing increased deployment. With the advent of ransomware, tape is one of the only storage platforms that provides an air gap from ransomware. This means that, while data on disk is being encrypted, the data backed up and archived to tape remains safe and ready to allow a recovery of an organization’s IT platform.  Cloud Storage Comes into its Own
Over the last decade, as cloud has gone from a discussion to a reality, more and more companies look to the cloud for their “IT solution”, whatever that solution may be. In some cases, it is moving from a Capex to an Opex model, and in other cases, it is the idea that cloud must be cheaper than on-premise storage due to scale. For customers who have jumped head-long into cloud storage, most have discovered the bitter pill that charges and other hidden fees drive the cost of cloud storage way up beyond their expectations and higher than their original budgets. It is not to say that storing data in the cloud is wrong or bad, it is that so many companies have jumped on the cloud bandwagon without doing the same due diligence required for any onsite architectural changes. Like getting caught up in a wave of excitement without much thought, some users ran to the cloud with their data in hand, only to now learn the true cost of that move. Cloud storage, for customers who have used it, still have it or have left it, now understand the nuances of a cloud contract and cloud pricing model (including the inevitable and undesirable ‘cloud lock-in scenario’) vs. a capital purchase.

Those users will be more cautious and careful over the next decade as they enhance their infrastructures, perhaps with a hybrid cloud approach that offers the best of both worlds – the scalability and accessibility advantages of cloud with the control, affordability and flexibility of on-premise storage.