1. Managed MySQL is here. Amazon RDS allows you to run MySQL on their hardware. It isn't perfect, but I think this is a great first release. I expect this will support PostgreSQL soon given that the command-line tools are not MySQL specific.

    Note:
    • This uses MySQL 5.1.38
    • I did not see an option to enable SSH connections to MySQL. I think that is required for this to be a great way to run MySQL.
    • This supports MyISAM and InnoDB. They don't give you command line access to the machines, so you cannot run myisamchk to recover corrupt MyISAM tables, nor can you run myisampack to compress them. I think it is a good idea to stick with InnoDB and then ask Amazon to upgrade to the InnoDB 1.0.4+ plugin.
    • This appears to use network attached storage for most data. For example, innodb_data_home_dir=/rdsdbdata/db/innodb. I am not sure whether this buffers data in the OS buffer cache and if it does not, that will hurt MyISAM performance as it does not buffer table data.
    • Replication is disabled. That makes it much easier to run many instances of MySQL in the environment. Replication state is not crash proof and Amazon probably does not want to spend their days recovering/replacing/rebuilding slaves. But that also limits the use of this for read scale out. Maybe Amazon and RightScale have something in progress to change that without introducing manageability overhead.
    • The master user does not have SHUTDOWN, SUPER or replication privileges.
    • Binlogs are enabled, but the master user does not have privileges to run SHOW MASTER STATUS. The documents state that databases can be recovered up to the last 5 minutes. I assume this means that any writes done are guaranteed to be archived somewhere after 5 minutes. If there were an option to archive the binlogs, then that would provide an extra degree of safety.
    17

    View comments

  2. Be careful when using FLUSH TABLES WITH READ LOCK (aka FTWRL). I have written about potential problems that may occur when using FTWRL. Anyone who runs ibbackup or xtrabackup on a server that writes a binlog needs FTWRL to run as fast as possible with as few problems as possible, but that is not always the case. In its current form, you must monitor FTWRL and either kill it or long-running queries when FTWRL takes too long.

    MySQL does three things when processing FTWRL. First it sets the global read lock. Then it closes open tables. Finally it sets a flag to block commits. You will have problems in production when FTWRL doesn't return quickly. It doesn't return quickly when there are long running queries as it waits for the current queries to finish. The problem is that insert, update, delete and replace statements are blocked after the first step. When FTWRL lingers in the second step (close open tables), then your server will stop accepting writes. An additional problem is that for deployments with many open tables, it is a lot of work to close and then re-open them. I need to confirm whether re-open is done serially because a mutex is held and whether InnoDB re-samples all indexes on all reopened tables to get optimizer statistics.

    I blame MyISAM for the current problems. As I am not a MyISAM expert, this is an educated guess and I welcome your feedback. The problem with FTWRL is FT (flush tables) and MyISAM is the reason that tables must be flushed. The --delay-key-write option and possibly other features in MyISAM allow open tables to buffer committed changes. The buffered changes are written to MyISAM data files when the open table is closed.

    INSERT DELAYED might also cause problems, but anybody who needs a hot backup shouldn't be using that option.

    I think we can make this better and my solution is DFTBGRL (don't flush tables but get read lock). Maybe it needs a better name. DFTBGRL skips the second step of FTWRL -- it sets the global read lock and then it sets a flag to block commits. This should be much safer to use in production.

    Update:
    After I wrote this, Harrison and Konstantin from MySQL/Sun gave me advice on a better way to fix this. I have implemented their advice and it appears to work, but I need to test it. The result will be much better than FTWRL for InnoDB.
    5

    View comments

Loading