Good post by Chris Mellor (thank you) over at El Reg
Facebook has ditched RAID and replication for its nearline storage, using distributed erasure coding to isolate what it calls “warm BLOBs” instead.
- BLOB — Binary Large OBject — Facebook user’s photos, videos, etc.
- Warm — data that has to be kept and is accessed at a lower rate than hot data, but more than archived or cold data. Typically, it’s more than a week old. Hot BLOBs, of course, are accessed more frequently.
- Erasure coding — the adding of calculated parity values (Reed-Solomon codes) to a string of bytes, such that the string can be recovered if an error deletes or distorts some of the complete string. Typically more efficient than RAID at protecting data as it uses less space.
Read on here