Millions of tiny databases

Millions of tiny databases

  • May 7, 2020
Table of Contents

Millions of tiny databases

The Physalia configuration store for chain replication of EBS is implemented as key-value stores maintained over a large number of these cells. They built a test harness, called SimWorld, which abstracts networking, performance, and other systems concepts. The goal of this approach is to allow developers to write distributed systems tests, including tests that simulate packet loss, server failures, corruption, and other failure cases, as unit tests in the same language as the system itself.

In this case, these unit tests run inside the developer’s IDE (or with junit at build time), with no need for test clusters or other infrastructure. A typical test which tests correctness under packet loss can be implemented in less than 10 lines of Java code, and executes in less than 100ms.

Source: blogspot.com

Share :
comments powered by Disqus

Related Posts

Taming ElastiCache with Auto-discovery at Scale

Taming ElastiCache with Auto-discovery at Scale

Our backend infrastructure at Tinder relies on Redis-based caching to fulfill the requests generated by more than 2 billion uses of the Swipe® feature per day and hosts more than 30 billion matches to 190 countries globally. Most of our data operations are reads, which motivates the general data flow architecture of our backend microservices.

Read More
Testing sync at Dropbox

Testing sync at Dropbox

Executing a full rewrite of the Dropbox sync engine was pretty daunting. (Read more about our goals and how we made the decision in our previous post here.) Doing so meant taking the engine that powers Dropbox on hundreds of millions of user’s machines and swapping it out mid-flight.

Read More
Scaling Driver Compliance across Lyft

Scaling Driver Compliance across Lyft

A passenger opens the app, requests a ride, and just a few minutes later there is a car with a friendly driver in front of them. They may know this driver went through a background check and other vetting; however, many might not realize all the complexity in the on-boarding process. Driver and vehicle requirements vary substantially across markets (and even at very proximate locations) due to varying market dynamics and government regulations.

Read More