Storage: A Low-Cost Decentralized Solution to Remote Data Backup and Storage

Cloud-based storage is becoming a popular way to offload the maintenance of backups and long-term archival.   Enterprise-grade cloud-based backup services are offered by companies like Microsoft, IBM and Oracle.  Nontraditional vendors like Amazon and their S3 storage is competing in this area is a big way.  Smaller players targeting small businesses and individuals include Box.net, Mozy, Sugarsync, Zetta.

All these vendors provide centralized backup solutions to the cloud.  Data stored using cloud storage vendors will typically store data across multiple servers and possibly keep the data in multiple data centers for redundancy, but one company is taking cloud-based storage into an interesting direction.  A company called Symform is offering cloud-based storage at low prices using a Napster-like decentralized file sharing approach.

Symform is based on the idea that large amounts of unused storage exist in data centers.  Symform provides a solution that harnesses unused storage space from the data centers of its members.  It also relies on some amount of storage being kept on Amazon S3.  Symform explains the method they use to process and store the data as follows:

Once a file is detected and scheduled for upload, it is first broken down into 64 MB blocks. Each block is encrypted using military grade encryption and using an independent key.

Each block is then further subdivided into 64x 1MB fragments. For redundancy we add 32 parity fragments using a RAID-like algorithm. We now have a total of 96 fragments of which any 64 fragments can be used to reconstruct the original block… All of this is happening behind the firewall, and data never leaves the premises without first being encrypted and encoded in this way.

Kevin Brown, vice president of Symform, said that “as you distribute your data to other systems around the world, other data in encrypted fragments from other users are sent to your contribution folder… With other applications, a business’ data is encrypted and sent to a single location, Brown said. “But with one location, there is a risk of compromising the data,” he said. “And since we use 1-MB fragments, downloads and uploads are faster than normal.”

Symform calls the system the Symform Cooperative Storage Cloud.  Storage prices are nearly a tenth of what other cloud storage vendors charge.  It’s an interesting and cost-effective option, but some people have expressed concerns over the safety of their data being distributed across the storage units of other Symform members.  Many organizations have a hard enough time trying to come to terms with storing their data within the remote data centers of a single cloud storage vendor, much less storing your data on the disk drives of 33 other organizations whose identity you do not know.  Others worry that while the storage fees are lower there will be additional bandwidth costs involved with incoming and outgoing data being written and read from the storage units being used by Symform.