M3: Uber’s Open Source Large-Scale Metrics Platform for Prometheus

M3, Uber’s open source metrics platform for Prometheus, facilitates scalable and configurable multi-tenant storage for large-scale metrics. To facilitate the growth of Uber’s global operations, we need to be able to quickly store and access billions of metrics on our back-end systems at any given time. As part of our robust and scalable metrics infrastructure, we built M3, a metrics platform that has been in use at Uber for several years now.

M3 reliably houses large-scale metrics over long retention time windows. To offer others in the broader community these benefits, we decided to open source the M3 platform as a remote storage backend for Prometheus, a popular monitoring and alerting solution. As itsdocumentation states, Prometheus’ scalability and durability is limited by single nodes.

The M3 platform aims to provide a turnkey, scalable, and configurable multi-tenant store for Prometheus metrics. As part of this effort, we recently releasedM3DB, the scalable storage backend for M3. M3DB is a distributed time series store and reverse index with configurable out-of-order writes.

Additionally, we are open sourcing M3 Coordinator, a Prometheus sidecar which provides a global query and storage interface on top of M3DB clusters. The M3 Coordinator also performs downsampling, as well as ad hoc retention and aggregation of metrics using retention and rollup rules. This allows us to apply specific retention and aggregations to subsets of metrics on the fly with the rules stored in etcd, which runs embedded in the binary of an M3DB seed node.

Source: uber.com