Sharding a database, twice
by Abigail -
Sharding a database, twice aimed at Any and is held in English.
There comes a time in the life time of a database, the database takes too many resources (be it disk space, number of I/O transactions, or something else) to be handled by a single box.
Sharding, where data is distributed over several identically shaped databases is one technique to solve this.
For a high volume database I used to work with, we hit this limit about a dozen
years ago. Then we hit the limit again two years ago.
In this talk, we will first discuss how we initialized switched our systems to make use of a sharded database, without any significant downtime.
Then we will discuss the totally different way we resharded the second time around, making use of the code infrastructure changes of a dozen years ago, and making use of the improved database tooling available.