Mydumper is a useful tool. Unlike mysqldump, it can take logical backups using many threads. Logical backups are frequently the only option for projects to migrate into the cloud, or for consolidating datasets from multiple servers. When you have many TBs worth of data to move, mysqldump is not a viable option.

However, mydumper does need a bit of finessing on occasion and I would like to talk about one of those occasions in this post. In terms of compression, mydumper comes with its own gzip compression which is quite slow and often is a bottleneck. Recently, it added zstd compression which is amazing, but try as I might, I could not get it to work as I would have liked. The way compression is done seems to suffer from a bottleneck in the pipeline.

Our client was, quite rightly, expecting better from our MySQL support, so we weren’t about to just wait out that bottleneck. We can do better than that. Luckily, we have ZFS compressed datasets with zstd. This was extremely useful, because it means that compression could be offloaded to zstd in another layer and that layer can handle it in an asynchronous multi-threaded way.

So after setting mydumper to dump the multi-TB database to a compressed ZFS partition, these were the performance results I was able to get:

MOAR POWER, IGOR!