Tuesday, March 16, 2010

Can a protocol be GPL?

I am trying to understand the behavior of MYSQL_OPT_READ_TIMEOUT which can be used to set a client-side read timeout for connections to a MySQL server. This determines how long a client will wait for a response to a request. I was uncertain based on the documentation.
The timeout in seconds for attempts to read from the server. Each attempt uses this timeout value and there are retries if necessary, so the total effective timeout value is three times the option value.
I read the code. The documentation is correct. The code attempts to read from the socket three times. I prefer to not have to multiply by three to know the real timeout. But it is too late to change this. Maybe they could add a new option -- MYSQL_OPT_READ_TIMEOUT_DO_NOT_MULTIPLY_BY_THREE.

I encountered this interesting claim while reading the source in sql/net_serv.c. We recently discussed this elsewhere. I wasn't aware that the claim is still in the source code.
This file is the net layer API for the MySQL client/server protocol, which is a tightly coupled, proprietary protocol owned by MySQL AB.
@note
  Any re-implementations of this protocol must also be under GPL, unless one has got an license from MySQL AB stating otherwise.
UPDATE

The read timeout is enforced by my_real_read in sql/net_serv.c. This code is hard to read. Output from the preprocessor is slightly better. I think it is an accident that the read is retried 3 times for client side code.

I am pretty sure that Drizzle removed this code. Good for them.

The outermost loop should be ignored as all retries occur in the first iteration of it. The comment that the first read is done with non blocking mode is wrong when this is used in the client library. That may explain why one of the retries is done:
    for (i=0 ; i < 2 ; i++)
    {
      while (remain > 0)
      {
        /* First read is done with non blocking mode */
        if ((long) (length= vio_read(net->vio, pos, remain)) <= 0L)
        {
After that fails on the first read attempt this block of code runs and then continue is called to jump to the start of the while loop. This comment is again wrong as the code within this block changes the socket to use blocking mode. The comment also contradicts the previous comment mentioned above.

          /*
            We got an error that there was no data on the socket. We now set up
            an alarm to not 'read forever', change the socket to non blocking
            mode and try again
          */
          if ((interrupted || length == 0) && !thr_alarm_in_use(&alarmed))
          {
            if (!thr_alarm(&alarmed,net->read_timeout,&alarm_buff)) /* Don't wait too long */
            {
The block above is not executed after the second read call fails because thr_alarm_in_use is true. In this case alarms aren't really used, the alarmed variable is an int set to 0 at function entry and set to 1 after the first read fails. This block of code is executed after the second read call fails and it executes a continue statement to branch to the start of the while loop and retry the read call for a third time. It increments retry_count to 1 before doing so.
          if (thr_alarm_in_use(&alarmed) && !thr_got_alarm(&alarmed) &&
              interrupted)
          {                                     /* Probably in MIT threads */
            if (retry_count++ < net->retry_count)
              continue;
#ifdef EXTRA_DEBUG
            fprintf(stderr, "%s: read looped with error %d, aborting thread\n",
                    my_progname,vio_errno(net->vio));
#endif /* EXTRA_DEBUG */
          }
The previous block of code is also executed after the read fails for the second time. However, retry_count was previously incremented to 1 and net->retry_count equals 1. So the continue statement is not called after the third read failure and my_real_read returns.

17 comments:

  1. i've filed a bug about the comment: http://bugs.mysql.com/52107

    ReplyDelete
  2. The whole "the protocol is GPLed" is FUD originating from old MySQL Inc, based on willful misunderstanding and a desire to sell licenses. Brian Aker addressed this in his recent blog post about MySQL and the GPL.

    If license follows network protocol as the linkage, then it would be illegal to access an IIS server from Firefox, or to access a GPLed web server from Explorer.

    To just completely bypass the issue entirely for people who felt obligated to worry about license issues with libmysql.so, there is now libdrizzle.so, which, in addition to speaking the new Drizzle DB wire protocol, also speaks the old MySQL wire protocol. libdrizzle is BSDed.

    ReplyDelete
  3. libdrizzle is awesome -- http://launchpad.net/libdrizzle

    ReplyDelete
  4. Mark; when monty wrote in pre 2000 and I re-wrote the protocol to some extent during my 4.1 release along with prepared statements support; the timeouts (net_read/write_timeout) is supported in both the ends and there is a net_retry_count; which dictates the behavior.

    The protocol(only TCP/IP) already supports the behavior what you requested. Are you certain that it does not work for you ? Look at all the vio_* functions.

    The question:
    Are you looking at the client end or server end ?

    ReplyDelete
  5. Here is the code piece that gets initialized in client end..

    void my_net_local_init(NET *net)
    {
    net->max_packet= (uint) net_buffer_length;
    my_net_set_read_timeout(net, CLIENT_NET_READ_TIMEOUT);
    my_net_set_write_timeout(net, CLIENT_NET_WRITE_TIMEOUT);
    net->retry_count= 1;
    net->max_packet_size= max(net_buffer_length, max_allowed_packet);
    }

    ReplyDelete
  6. The timeout as measured by an application will be 3 times the value of the argument for MYSQL_OPT_READ_TIMEOUT when mysql_options is used. This is correct per the documentation but yet another sharp edge that makes MySQL more expensive to use.

    After tracing the code run on the MySQL client I think the 3 retries are an accident rather than put there by design.

    As I describe above, comments in the code are wrong. That is somewhat understandable given the inscrutable nature of this code.

    ReplyDelete
  7. More on this matter in my blog.
    http://datacharmer.blogspot.com/2010/03/protocol-gpl-and-how-bazaar-can-help.html
    The bottom line is that a protocol should not be protected by the GPL.

    ReplyDelete
  8. Hi Mark! Good spotting! Would you mind submitting a bug report about this, so we can investigate and fix this? Cheers :)

    ReplyDelete
  9. Someone else did that first. All of this is good news as the MySQL implementations of the protocol don't need special protection. They do great when competing on ease of use and technical excellence.

    ReplyDelete
  10. The bug is http://bugs.mysql.com/bug.php?id=52107

    ReplyDelete
  11. Thanks Mark! I was not referring to filing a bug about that bogus GPL claim, but rather about your other observation (the 3 retries being an accident). :)

    If you consider this a bug that deserves to be fixed, that is...

    ReplyDelete
  12. Lenz - I think it is too late to change that as it is documented and many people might be surprised were it to change. I encountered this while debugging performance hot-spots created by a too large value for innodb_lock_wait_timeout. When the client-side read timeout expires and then the connection is closed, a session on the server stuck in an InnoDB row lock wait does not notice this until it wakes from the lock wait. It won't even respond to KILL until the lock wait timeout expires. A separate feature request was opened for that.

    ReplyDelete
  13. For those who want to subscribe to it a bug (or feature request) is open for connections in InnoDB lock wait not responding to kill -- http://bugs.mysql.com/bug.php?id=51920

    ReplyDelete
  14. Thanks for the explanations and for submitting that feature request, Mark.

    ReplyDelete
  15. Mark, anything that blocks KILL from working has in the past been treated as a bug that needs to be fixed quite quickly. Can't leave people having to use kill -9 on a production server to restore service and preventing that is KILL's job.

    ReplyDelete
  16. My favorite code keeps on getting better. 11X is way better than 3X -- http://bugs.mysql.com/bug.php?id=52633

    ReplyDelete

 
Creative Commons License
This work is licensed under a Creative Commons Attribution-Share Alike 3.0 United States License.