Economics of Tape Indicate Warm Waters for Glacier: Why pay 10x more to Control the Data Growth Tsunami?
I'm excited to hear about Amazon's new archive service, Glacier. I am sure some of my own data will end up in Glacier some day. With the tsunami of data growth, everybody needs data storage options.
The part of Amazon's announcement that missed the mark is their claim that Glacier is the death of tape. I think Glacier will be successful – but I don't see that its success (or I guess possible failure) has much to do with the tape market. Glacier is clearly designed for smaller data sets and to be a very deep, very cold archive. Tape's largest growth market is for large data sets and in active archive where data is indexed and accessed frequently.
A few examples of markets relying on tape-based active archive applications include:
Media & Entertainment (M&E) – M&E users typically archive up to petabytes of images and video files, and utilize their archive data frequently, actively querying it and retrieving clips. It is not practical for them to use the cloud, which restricts when and how they move and use their data. And the costs to use the service would be staggering. M&E users need to access their archive data much more often – and rapidly – than the Amazon service is designed for.
New archive markets for tape are emerging, with data requirements that are very similar to those in M&E.
- High Performance Computing (HPC) -The HPC folks have multi-petabyte data sets that are being created very rapidly. This data needs to be actively indexed and rapidly retrieved. Yesterday's petabytes of archived data are now expanding to exabyte levels. I moderated a panel at the Super Computing conference last year with several national labs and an analyst. The consensus from all was it was much more practical and affordable to "move the compute to the data than the data to the compute" or stated another way, cloud is not practical for an HPC data archive.
Established and emerging markets with big data use tape, not cloud storage. Some of the reasons:
- long term storage economics of tape.
- eliminates the requirement of large and expensive network connections to the cloud.
- power efficiency of tape.
- fast ingest and retrieval times (streaming speeds as fast as or faster than any disk technology with just a 60-second delay for tape mount).
- security through integrated AES-256 encryption and the ability to keep the data onsite and in the hands of the company that owns it.
- tape's data integrity advantages.
Amazon Glacier is not practical for active archive markets with up to exabytes of data. Tape is. Bandwidth costs are too high, retrieval time is too slow and the Glacier model doesn't meet the frequency of data access requirements. Just think about retrieval times: it would take about 40 days to move one (1) petabyte across an OC48 and 10 days to move it across 10 Gb/E. When talking about large data archive, local storage is a must.
Additionally, the economics of Glacier are not competitive with tape. Glacier's pricing is being promoted it as low as $0.12 per GB per year. The comparative costs for a Spectra TFinity tape library are $0.0008 (cost per GB per month amortized over 5 years). Add in power, floor space and personnel costs for all 5 years and the total cost should still be well below $0.01 per GB per year for the period.
The configuration in this example includes everything (encryption software, media, drives, installation, warranty, etc). This figure is based Spectra's average list price for a TFinity and includes zero hidden costs.
10,000 slots (capable of going to over 11,000 slots for the frame count)
10,000 LTO5 media
12 LTO5 drives
Pro Encryption (to protect the archived data)
Power cost = $.08 / kwH
Utilization (duty cycle) rate = 80%
Floor space cost = $100 / sq. ft.
1 Storage Admin (high end of pay scale + benefits, estimated from on-line salary survey)
When multiplied to terabytes and beyond, the savings of $0.01 per GB per year storage, compared with Glacier's $0.12 per GB per year, are massive. These savings using a Spectra TFinity are just one example of savings through using tape. I saw an article from Quantum today saying they can offer 10PB of tape stored for 5 years for $669,663.70 versus around $6,000,000 for Glacier. It appears Glacier is at least 10x more expensive than the price multiple tape companies can offer for on premise tape archive storage. These are staggering numbers! Yes, Glacier will probably do well. AND the tape market will continue to grow.
Follow me on Twitter @MollyRector.