Sunday, May 25, 2008
Q4M and the need for XA
Q4M is a new storage engine that supports persistent queues. For a project that claims to have been started in December 2007, the code and features are impressive. But one thing is missing. I want to perform a dequeue and update InnoDB in one transaction. MySQL could use XA internally to support this, as XA is already used on the master between the binlog and storage engines. However, XA is not supported to group changes between multiple storage engines into one transaction. Until now, there has not been a demand for that feature. Q4M may create that demand.
Subscribe to:
Post Comments (Atom)
Posts
Mark, XA is supported to group changes between multiple storage engines into one transaction - that's the whole idea of two phase commit. One tiny details - it is used if all storage engines affected by the transaction support XA. Binlog pretends to be an XA-capable storage engine, InnoDB is XA-capable storage engine. That's why XA is used for them. When Q4M will support XA, two-phase commit will be used for it automatically. At least that's how it should work, bugs are possible of course.
ReplyDeleteThen I don't understand the MySQL documentation on the restrictions on XA. The claim is that internal XA is not supported. But if the support is there, then this is great for apps that want to perform queue based transaction processing.
ReplyDeleteAccording to bug 32926, only one storage engine that supports prepare can be running at a time. That will prevent internal XA from being used to commit a change between Q4M and InnoDB in one transaction. I assume I could do XA between two servers, but even if I were willing to do that, and I am not, external XA doesn't write changes to the binlog for transactions that are prepared before a server crash and committed after recovery.
ReplyDeleteI've no idea why documentation says that. Internal XA is supported, but of course it's limited by the capabilities of the storage engines.
ReplyDeleteBUG#32926 talks about two different things. An assert (quoted in the bugreport) is not a bug, it is an intentional feature - InnoDB's "rollback everything on recovery" behavior is backward compatible, but not compatible with XA, so I implemented it but added an assert to make sure it cannot be enabled if there's more than one XA-capable storage engine. The #ifdef must, of course be removed when a second XA-capable storage engine is added (falcon). But Ann also vaguely mentions "other problems" - I assume they're bugs, and should be fixed as such. As I said in my first comment, bugs are possible.
Thank you for your interest to Q4M.
ReplyDeleteI definitely aggree with you that supporting transaction between InnoDB and Q4M would be the way to go.
In January I asked if it was possible on the MySQL internals mailing list (http://lists.mysql.com/internals/35269), and after hearing from Sergei that the related code within MySQL 5.1 might have bugs, I just had given the idea a low priority.
But now that Q4M has become more mature, it might be time to look into supporting transaction.