This event has ended. Visit the official site or create your own event on Sched.
Back To Schedule
Wednesday, July 13 • 09:20 - 09:40
Running Storage at Facebook

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

The mission of the Data Warehouse Storage team at Facebook is to run an HDFS deployment that stores hundreds of petabytes reliably and efficiently.

Due to its stateful nature, there are some unique challenges to operating a storage system. How do we take a machine out of production for repair without compromising data availability? What trade-offs do we make between replication strategy and data availability to make sure we get more bang for the buck? How do we ensure that colocated tasks that run on our storage nodes exploit available resources as much as they can without tipping hosts over?

In this talk, we'll share some of the lessons that we have learnt from running HDFS at Facebook. We will discuss our biggest operational challenges and we'll outline the evolution of the different solutions that we put in have in place over time. We will also introduce Warm Storage, a novel block storage system that we built at Facebook to replace HDFS, and we'll discuss how our learnings from HDFS have affected the design of Warm Storage.

avatar for Federico Piccinini

Federico Piccinini

Production Engineer, Facebook
Federico is a Production Engineer at Facebook and has been working for the past one and a half year on large scale block storage systems. Before Facebook, Federico help running the Storage infrastructure at Spotify. Likes: open source, large scale distributed systems and Broadway... Read More →

Wednesday July 13, 2016 09:20 - 09:40 IST