One of my peers described managing backups at scale. It didn't make it to planetmysql because of the way it was posted. It is worth reading.