The most popular and comprehensive Open Source ECM platform
Let us know more
Storage: Data Preservation is a Growing Problem
The King Tut exhibit has recently been at the de Young museum in San Francisco. The artifacts at the exhibit included statues and even wooden boxes that were amazingly well preserved after three thousand years. Obviously the dry desert Egyptian climate has much to do with enabling the artifacts to have remained in such great condition.
If only we could manage to preserve the world’s data as well as the Egyptians succeeded with King Tut’s remains and artifacts! Most of the data being created today is being stored on media with life expectancy of just a few of decades, at best.
While data storage technology is evolving to improve both capacity and performance, the problem is that data formats and storage hardware are being continually obsoleted and that media storage devices aren’t being built to last for extended periods of time.
So, while supermarket plastic bags need 1000 years to decompose in landfill sites, many of today’s storage media devices are nonfunctional in less than a decade. A NewScientist article estimates that some of today’s most popular media used for storing important data backups, like magnetic tapes and optical discs, are being stored in formats that are often outdated after only five years.
On the hardware side, optical discs constructed with a reflective layer of gold and a phthalocyanine dye look like the best bet for longer-term storage. These type of optical discs can last for a hundred years. But most other media hardware have lifetimes of five to ten years.
Another example is flash memory. Flash-memory drives have become extremely popular, but their design has focused on speed and capacity rather than stability. Flash drive makers estimate about a 10 year lifespan for their products.
The environment where the media is stored is a major factor in how long the media will ultimately survive. Heat and humidity are the main causes for media to rapidly degrade. To guarantee longevity of your data, you could invest in the infrastructure to maintain a perfect environment for preserving storage media, or then again you could ship your media off to a buried site somewhere in Egypt.
The biggest problem is that data storage technology is changing rapidly. For example, when scientists from NASA wanted to resurrect data stored by Lunar Orbiter missions from the 1960’s, it took a team of people and three months of time to figure out how to retrieve and use the data stored on a set of magnetic discs.
A solution to one piece of the problem might be to adopt something like ILM for long-term data archival. With ILM, frequently accessed data is stored in fast-access expensive memory, and the data is gradually migrated to cheaper storage media as the data is accessed less frequently or as the content of the data loses its value. ILM has the interesting property that data can flow easily from one storage media type to another.
A key point of ILM is that even though there are many types of storage media, all data remains accessible. With this kind of a system using interconnected storage media types, as storage is obsoleted, data can flow and be managed by newer types of storage technologies and eventually older technology can be emptied and retired.
The bottom line is that making a tape, putting it on a shelf and then waiting is a sure way for your data to quickly become unusable.
