Tuesday, October 14, 2008

Faith-based upgrades

MySQL documentation states that replicating from a 5.0 master to a 5.1 slave should work. This is very different from stating that it does work. That section of the manual should enter the 5.1 no-use case competition.
In general (but not always), slaves running MySQL 5.1.x can be used with older masters, but not the reverse.
Most of us upgrade to a new version by running it on slaves for a long time before putting it on a master. Can we put it on critical slaves? After reading the documentation I am not sure.

An Enterprise database must claim that it supports replication from one release to the next. MySQL seems to support it in practice -- I am able to replicate from 5.0 to 5.1. Am I taking a risk by using 5.1 slaves or does the documentation need an update?

10 comments:

  1. I would venture to guess they are just playing it safe. I think you can translate "should" to "it's designed to work, but we haven't evaluated every instance and we can be liable for anything that happens..."

    It worked fine for me. I moved data centers in June, and I upgraded from 5.0.45 to 5.1.24 in the process. The only problem I had was replication lag due to network bandwidth (long bin-log sync times).

    ReplyDelete
  2. Mark,
    the manual states that the exception is about masters using version 4.0. I asked the documentation team, and they confirmed that this is the only known case.
    The documentation could be more precise, but the manual page specifies what the "not always" part is about.

    Giuseppe

    ReplyDelete
  3. @Giuseppe -- The documentation needs to be updated to state that replication from the previous stable release to the current stable release always works or the documentation should clarify that this is not supported. It is very ambiguous today and seems to leave a lot of loopholes to justify when it doesn't work.

    ReplyDelete
  4. Gregory is correct.

    MySQL Replication is designed to be backwards-compatible, but there are some exceptions, which are noted in the documentation. See http://dev.mysql.com/doc/refman/5.1/en/replication-compatibility.html (which you already linked to) and http://dev.mysql.com/doc/refman/5.1/en/replication-features.html (which is linked to fairly prominently from that page).

    For us to say 'it always works' without qualification would be about as honest as stating 'you'll always find exactly what you're looking for as the first item listed in Google search results'. After all, everyone knows that all Open Source software is 100% bug-free and that Google can read minds, right?

    More seriously, we generally support replication from 'old' masters to 'new' slaves and any exceptions to this are either known issues or bugs. We make our best effort to document all known issues fully, and to fix bugs as quickly as possible.

    ReplyDelete
  5. @Jon - why is there so much uncertainty in the description of this? Other sections of the manual don't state that MySQL generally or should return the correct rows for a query.

    This has nothing to do with bugs. Every RDBMS has bugs that contradicts documented behavior.

    I have looked at the advice that MySQL provides. It expects me to make the determination. Do you really expect user's to benefit from this?

    From http://dev.mysql.com/doc/refman/5.1/en/replication-features.html
    "If you are planning to use replication between 5.1 and a previous version of MySQL you should consult the edition of the MySQL Reference Manual corresponding to the earlier release series for information regarding the replication characteristics of that series."

    If I restore a slave with 5.0, upgrade it to 5.1 and then replicate to it from a 5.0 master, everything should work.

    ReplyDelete
  6. Hi!

    This is one of the reasons, and perhaps he most major, that we are extending out the development time for the first release of Drizzle. I don't want to see us release and have any user question this with us.

    Cheers,
    -Brian

    ReplyDelete
  7. Hi,

    I have a 5.1.23-rc slave with 468 days uptime which is replicating from a 5.0 master. So yes, you could assume that it works in practice.

    ReplyDelete
  8. Hi Mark,

    I only became aware today that you'd filed a bug (http://bugs.mysql.com/bug.php?id=40023) concerning this issue, because it had been filed under the "Server:Replication" category rather than "Documentation". I've corrected this and have assigned the bug to myself. In future, should you see any issues with the Replication documentation, please file a Documentation bug - this is the fastest way to get it into my work queue.

    You take exception to the statement, "If you are planning to use replication between 5.1 and a previous version of MySQL you should consult the edition of the MySQL Reference Manual corresponding to the earlier release series for information regarding the replication characteristics of that series," and I'm afraid that I still don't understand exactly why.

    In fact, it seems to me that we'd be negligent *not* to include such a statement in the documentation. Here's why: We have at least 3 different audiences whose requirements we try to address with the MySQL Manual: (1) Customers having MySQL Support contracts (this is you, Mark); (2) our own staff (MySQL Support team, QA, developers, and others at MySQL/Sun); and (3) GPL MySQL users who don't have support contracts. This third group may not be giving us any money, but they are *extremely* important to us - without them, we'd have nothing like the ubiquity that MySQL enjoys today.

    As a commercial MySQL customer, your employer has a contract with mine that says, "/X/ will work for you - and if for some reason it doesn't, we'll *make* it work for you," and you can contact our Support staff and expect an answer to any issue you might encounter within a stated period of time.

    However, our GPL/gratis users don't have this luxury - they're providing their own technical support; we try to streamline this task for them by providing as much information as we can in the MySQL documentation, and because this documentation is quite extensive (2500+ pages in the MySQL 5.1 Manual), we also try to provide plenty of pointers to related information where it might not be obvious for them to look.

    So we're not expressing uncertainty - it's not the purpose of the MySQL Manual to "express" anything. The purpose of the MySQL Manual is to provide accurate technical information to users, especially to those users who can't just pick up the phone and tell us to solve their problems for them.

    These things being said, I've discussed your issue with MySQL Support, the Replication Team lead, and others, and I'm reviewing the portions of the documentation with which you have issues. I intend to complete this review and act on it within the next couple of days.

    Thanks for your input, and thanks for using (and blogging about) MySQL!

    ReplyDelete
  9. @Jon - the documentation is excellent and reflects the state of the software. I think that upgrading to a new MySQL version is more complex than it should be and this complexity is reflected in the documentation. It is hard to make sense of all the restrictions that may cause problems during an upgrade. Some of this cost would be reduced if MySQL provided a tool to scan binlogs and determine whether there were any potential problems based on the contents.

    ReplyDelete
  10. Hi Mark,

    I'm starting to see what you mean about how some of the replication material is/was being presented.

    I see now that you actually weren't the one who filed Bug #40023, so perhaps you might like to subscribe to it?

    I've committed some changes to http://dev.mysql.com/doc/refman/5.1/en/replication-compatibility.html (as well as the 4.1, 5.0, and 6.0 versions of this page). These changes are now online, and I would appreciate your feedback - you can just add your comments to the bug report or drop me an email (jon at mysql dot com) if you like.

    I'd also like to hear any other ideas you might have regarding the replication-features and related sections Replication docs.

    cheers,

    jon.

    ReplyDelete

 
Creative Commons License
This work is licensed under a Creative Commons Attribution-Share Alike 3.0 United States License.