1. There have been interesting discussions in the PostgreSQL community about adding support for index only scans. On several occasions people were curious about how InnoDB supports this. A recent post by the InnoDB team is an excellent overview. A brief summary of that post and other material is:

    • records in the clustered (primary) index store hidden columns (DB_TRX_ID, DB_ROLL_PTR)
    • records in the non-clustered (secondary) index do not store hidden columns
    • records in clustered and non-clustered indexes have a delete-mark flag
    • records are not updated in the secondary index, they are delete-marked on delete, inserted on insert and delete-marked/inserted on update
    • delete-marked records are removed from indexes by the purge thread when it is safe to do so
    When a secondary index page is read, if the max transaction ID on the page is less than the max transaction ID for which all transactions are visible to the reading transaction (low-water mark, up_limit_id), then the page can be used as is and the page read is index-only. If this condition is not true, then for any entry read from this page the record is read from the clustered index page to determine whether the index entry is visibile. In that case the secondary index read is not index only. Index only matters because when things are not index only there can be an additional random disk read to the clustered index for each entry read from the secondary index.

    The max transaction ID for which all transactions are visible to the reading transaction is described as the low-water mark and assigned to the up_limit_field in the read view (read_view_struct). This is the max transaction ID for which there are no unresolved transactions when the reading transaction starts. If there is a long-open transaction when the reading transaction starts, then up_limit_id will be less than the transaction ID of the long-open transaction.

    I began to read the code for this today as I want to add a counter for the number of secondary index page reads that are and are not index only. If you want to read the code too the function lock_sec_rec_cons_read_sees determines whether all entries on a secondary index page are definitely visible to a transaction (read view).

    If you are interested in this topic, I recommend these books:

    5

    View comments

  2. I make bad jokes about Monty-style code. I don't like to read or modify it. I love to run it in production. It never crashes. It never leaks memory. I exaggerate a little bit but not too much. It is remarkably stable.

    This is an amazing accomplishment. Alas some of the senior developers who did that at MySQL have since left. Another senior developer left this week. I hope this trend does not continue.

    Everyone can be replaced. Smart people can be found. But smart and productive people are not as widely available and it takes a while to figure out how things are done in MySQL. I know because I have made a lot of mistakes while trying to make things better.

    Perhaps the Google response is appropriate. Everyone gets a 10% raise and a $1000 holiday bonus.

    I did not submit this to http://planet.mysql.com
    3

    View comments

Loading