Part one of three – Introducing the Two-Tier Model in M & E
Content is the lifeblood of media and entertainment organizations. Content growth, ever increasing raw and edited file sizes and greater resolutions — 4K and 8K formats, virtual reality, augmented reality and high dynamic range — have placed tremendous pressure on storage systems. The modern media organization demands collaboration like never before, meaning timely multi-site access to assets is crucial to success. Furthermore, content workflows rely on efficient analysis and search capabilities, with organizations often requiring multiple workflows within a single infrastructure, exceeding the functionality of the traditional filesystem. Increasing scale, level of collaboration and diversity of workflows are driving the industry toward a new model for media storage.
In the past, storage usage was defined by the technology leveraged to protect digital assets using a pyramid structure, with the top of the pyramid designated for SSD to store ‘hot’ assets, SATA HDDs used for ‘warm’ storage and tape used for the bottom of the pyramid to archive ‘cold’ content. Today, Spectra proposes a new two-tier architecture to replace the aging pyramid model.
The new two-tier paradigm, recently introduced by Spectra in its “Digital Data Storage Outlook 2019” report, focuses on the usage of the assets rather than the technology. The new paradigm combines a project tier where in-progress data resides, which is file-based, and a second or perpetual tier where finished and less frequently changed data resides, which is object based. Content moves seamlessly between the two tiers as assets are manipulated, analyzed, shared and protected.
The project tier leverages the file system’s high bandwidth, high-performance random I/O and ability to write to any location within a file to manage dynamically-changing assets and workspaces. This is sometimes called the primary tier or production tier. A media production company, for example, may use this tier to capture raw camera streams, audio streams and timecode simultaneously. Players in this space include the likes of Alcestor, Qumulo, Stellus Technologies and Pure Storage. They facilitate this project tier with their file-based production storage systems.
The perpetual tier relies on object storage’s use of HTTP to maintain a simple, high-level interface that can operate at scale, send an object in multiple parts and retrieve a range of an object. The technology enables encryption and access controls (Uniform Resource Identifier or URI) for safe sharing of creative assets across a wide network and pre-staging of objects at remote locations. The static nature of objects further allows them to be cached in global content delivery networks for cloud-based distribution of completed files and their respective descriptive manifest, such as MPEG-Dash as used by YouTube and Netflix. A good example of this is Spectra’s BlackPearl, an object storage platform that moves content seamlessly and economically into disk, tape and cloud.
A critical component of superior content storage technology is its ability to deliver shared access to data across multiple users, applications and edit systems. Spectra proposes that instead of a pyramid, defined by the type of storage technology, the new model focuses on the use cases of storing the content. In one case, where production occurs, one type of storage may be used. Where assets are not in production, but persist, the assets can reside in a perpetual tier. To simplify, production storage tiers are built on filesystems and perpetual tiers are built on object storage systems.
Stay tuned for more on this topic in our upcoming blogs over the next week. Part two will dive deeper into file-based production systems and the specific features and technology that differentiates solutions.