Review - Glacier: Highly Durable, Decentralized Storage Despite Massive Correlated Failures

Glacier is a decentralized storage system that attempts to provide integrity
guarantees in the event of large-scale correlated failures. The authors
argue that the assumption of failure independence is not realistic in
practice. Massive replication is employed to ensure that all data can be
recovered even when up to 60% of nodes have failed. This solution targets
the corporate environment specifically, which creates favorable conditions.
That is, in a corporate environment nodes are expected to be active most of
the time. Specifically, the lifetime of a node is on the order of weeks or
months, and sessions last on the order of hours or days.

The Glacier team have used this storage system in practice with ePOST and
FreePastry (developed by the same team). They claim that the system has
survived several kernel panics, JVM crashes, and in one case half of the
nodes became disconnected. While these error are cute, they do very little
to stress the system. Note that in none of these cases data loss occurred.
Further, had data loss occurred, Glacier maintains 48 fragments for each
file with an erasure code of 5. As there were only 35 node, it would have
been nearly impossible to observe data loss.

There are many problems with the Glacier use case. While I believe that this
research project presents many challenging systems design issues, there is
no practical purpose for this model. Most obviously, the cost of deploying
and maintaining such a system cannot possibly justify its limited use,
especially when a common RAID would perform just as well, or better. Also,
it is obvious that massive replication will produce high availability.
Granted, the use a erasure coding does reduce the replication storage size,
but this was not a contribution of the paper.
Received on Thu Nov 24 2005 - 09:13:55 EST