By Jenna Blumenfeld
New Zealand has one of the most dynamic weather systems in the world. Shaped by the collision of polar and subtropical air over a varied landscape that contains mountains, coastlines, and forests, the country’s weather is famously unpredictable.
It’s why New Zealanders often say you can experience “four seasons in one day.”
This atmospheric complexity makes forecasting hazardous conditions like cyclones and floods a constant challenge for meteorologists. Earth Sciences New Zealand (ESNZ), the country’s government-funded research organization, is working to improve the accuracy of forecasts with Cascade — a supercomputer capable of computing speeds of 2.4 petaFLOPS and equipped with advanced AI capabilities.
Designed to simulate everything from seismic activity to tsunami predictions to rainfall, Cascade aims to bolster New Zealand’s resilience against natural disasters while helping researchers better understand shifting climate patterns. But to realize the full potential of Cascade, ESNZ needed a high-performance archive that could manage petabytes of research and seamlessly support data-intensive HPC workflows.
Here’s why ESNZ turned to Spectra Logic for help.
A Robust Data Archive from Spectra
“Our biggest problem was handling all the data,” explains Jeff Zais, HPC Senior Science Advisor & Platforms Architect at ESNZ. “We didn’t have a good solution to take our data, put it into the archive, and bring it back out efficiently.”
Before adopting Spectra, the organization relied on an aging IBM TS3500 tape library to store data. As it reached end-of-life — and Cascade’s commissioning loomed on the horizon — the need for a new storage solution became critical.
To complement the capabilities of Cascade, ESNZ required a modern, scalable tape archive with a proven track record in high-performance computing environments. The Spectra Logic TFinity tape library delivered on all fronts.
Named Rapids, the data archive is designed to scale to 85 PB per TFinity with LTO-9 media. (In the future, ESNZ will switch to LTO-10, raising the capacity to 141 PB per TFinity.) It includes:
- 2x Spectra TFinity tape libraries, each with 4 frames and 4,700 slots
- LTO-9 tape media
- Versity ScoutAM servers
Currently, Rapids houses 15-20 PB of weather data dating back 25 years — a treasure trove of knowledge that Cascade will harness to forecast future weather patterns.
Training AI Weather Models
One of the most exciting promises of Cascade is its ability to deliver significantly more accurate weather forecasts using AI-ready GPUs.
As with any artificial intelligence system, the quality and quantity of data are central to its success. ESNZ scientists aim to train AI models using petabytes of archived data stored in Rapids, feeding the system with highly detailed past weather patterns.
“We have a team of AI experts that are pulling the old weather data out and working with it on our high-performance file system and GPUs,” says Zais. “They start with a global model and then identify how weather is going to evolve over certain regions in New Zealand.”
ESNZ is combining physics-based modeling with advanced AI to improve how it forecasts and understands the environment. This supports outcomes such as natural hazard warning and recovery, or the adaptation of farming systems to more variable and extreme conditions. Cascade can generate long-range forecasts with unprecedented precision. Such predictions can make a difference for regions of New Zealand that are on the front lines of environmental shifts, such as farming communities.
“What matters is increasing the level of detail in a way that is usable, forecasting accurately for specific locations, and maintaining that accuracy further out. That relies on having secure, scalable access to very large volumes of scientific data,” explains Warrick Johnston, Chief Digital & Technology Officer of ESNZ. “Our partnership with Spectra Logic is central to protecting and managing the nation’s data assets so they can be reliably used at scale, including for advanced AI-driven modeling. This gives us a much stronger basis to plan, protect people and infrastructure, and make decisions with more confidence.”
As the organization increases the resolution of its simulations and adds more AI GPU nodes to Cascade, data volumes will accelerate: ESNZ will potentially produce over 20 PB per year.
Core Partnerships
Key to the success of this initiative is a strong network of integrated partners. ESNZ’s collaboration with Versity, Xenon, and Spectra ensures the entire data pipeline — from input to archive to retrieval — operates smoothly.
“All of the vendors involved provided great support to get the solution integrated and operating,” says Zais. “We have Xenon architecting the system. We have Versity software that manages the data. And we have the Spectra tape archive as hardware. All three of these companies work together really well.”
This partner ecosystem ensures that ESNZ researchers can focus on science rather than infrastructure intricacies and that the Cascade supercomputer has the agility needed to thrive in a fast-paced environment.
A Backbone for Predictive Power
Spectra Logic plays a pivotal role in enabling Earth Sciences New Zealand’s important initiatives by delivering a scalable, high-performance archive tailored for Cascade. With the Rapids archive, ESNZ can securely store and retrieve weather data to train sophisticated models. Cascade can have the data backbone it needs to analyze New Zealand’s volatile climate and deliver faster, more accurate forecasts.
In a country where the weather can change by the minute, that level of predictive power is essential.
