Cephalocon 2019

While being a member of a team developing a distributed low-latency block-store, the author has been asked the question "why not just use ceph?" so often that he finally forced himself to understand some of the crush/rados basics.

It turned out, the block-store his team was working on was fully "declustered", while the ceph PG-indirection layer allowed to limit the level of "declustering".

PG concept is explained in various online sources but rather from a ceph configuration perspective. In this short talk, it is shown from a developer perspective on a very small example cluster what particular technical problem PG layer solves and how. Specifically, we illustrate the trade-off between cluster utilization during failure recovery and data loss probability due to coincident device failures and show how exactly PGs allow to favor the former or the latter.