Vaultaire: a data vault for system metrics, backed onto Ceph
Before you can do serious analytics you need to have serious data. You can't go averaging it away just to save space; you need the raw data.
Vaultaire is a distributed metrics store. It is fault tolerant, performs well, is space efficient, and — since it stores data directly in Ceph — scales horizontally as your cluster grows.
Most systems crumble when faced with the thousands of metrics per second typically being gathered in a production environment. Ceph is good at absorbing lots of writes, and is especially good at handling randomly distributed reads.
You may have experienced Ceph via it's S3-compatible HTTP gateway or via the RBD block device service. You may not know these are built atop a simple library that handles talking to the storage cluster and locating your data, librados, and that's how Vaultaire stores its time-series data.
Using librados directly opens some pretty cool possibilities. It's powerful, but low-level, and as you'd expect there are gotchas. We'll talk about building a librados-based application, lessons learned relying on Ceph in production, and the challenges of building predictive modelling tools on top of a metrics store that doesn't throw data away.