Review - Total Recall: System Support for Automated Availability Management

TotalRecall is a is a system that automatically measures the availability of
a distributed file system. Further, it attempts to predict the future
availability of the objects managed by the file system and repairs the
system to maintain a specified level of availability. The motivation for
this work is the observation that peer-to-peer networks are fragile to
storage concerns due to high churn. TotalRecall attempts to guarantee some
level of availability by predicting individual node availability, deriving
the redundancy requirements based on the predicted availability and
repairing the system based on those calculations. A key component is that
these repairs are preformed automatically.

TotalRecall suffers from the usual overlay pitfalls. For example: Why would
people donate storage space? What if I want 100% availability? What about
security? etc. However, more specific to this design, there seems to be a
lot of overhead communication. The paper quotes an average of 6.5 KB/s of
repair bandwidth per file (Section 6.2). A quick calculation shows that this
would result in 16 GB of data per month per file. If considering Roger Cable
Internet service in Toronto, this is approximately 25% of the allotted
monthly bandwidth. The authors also admit that the availability probing
solution that they have implemented does not scale (Section 4.6).

While the TotalRecall system attempts to be fine grained (maintaining short
term and long term information) there is no indication of how additional
statistical information could increase performance or reduce overhead. Could
the system also maintain statical metrics on host heterogeneity, and would
that help? The authors use a homogeneous system for their simulations
(infinite storage capacity, fixed latency, infinite bandwidth, etc.), but
fail to analyze how these assumptions effect the results.
Received on Thu Nov 24 2005 - 00:30:59 EST