linux.conf.au 2014

Perth, Western Australia - 6th to 10th January 2014

linux.conf.au


<-- Back to schedule

Distributed storage and compute with Ceph's librados

Project: Ceph

The Ceph distributed storage system sports object, block, and file interfaces to a single storage cluster. The core infrastructure supporting these use cases is a distributed object storage and compute platform called RADOS (reliable autonomic distributed object store). The low-level librados API exports a conceptually simple yet powerful interface for storing and processing large amounts of data and is well-suited for backing web-scale applications and data analytics. In features a rich object model (bytes, attributes, and sorted keys/values), efficient key/value storage (using leveldb on the backend), atomic transactions (including efficient compare-and-swap semantics), object cloning and other primitives for supporting snapshots, simple inter-client communication and coordination (ala Zookeeper), and the ability to extend the object interface using arbitrary code executed on the storage node as part of the IO pipeline. Librados classes are a plugin API that allows computation to be performed on the storage node, avoiding expensive read/modify/write cycles, while still leveraging the replication, atomicity, and high-availability provided by RADOS.

This talk will include a brief overview of the Ceph architecture, and then focus on librados, the API and how it is used, the security model, and some examples of RADOS classes implementing interesting functionality (like cls_lua, which embeds a LUA interpreter for arbitrary code execution).

Sage Weil

Sage is the chief architect of Ceph, an open source distributed storage system that may be storing much of your cloud data in a few years' time. Sage is currently founder and CTO of Inktank, an enterprise storage company dedicated to supporting organizations deploying Ceph.