Future work:
- Reduce the size of mutex and rw-lock structures
- Reduce contention on the sync array mutex
- Reduce contention on kernel_mutex
- Reduce contention on commit_prepare_mutex
- Reduce the number of mutex lock/unlock calls used when a thread is put on the sync array
- Name all events, rw-locks and mutexes in InnoDB to make contention statistics output useful
- Add optional support to time all operations that may block
- Introduce dulint to native 64-bit integer types
- Make BUF_READ_AHEAD_AREA a compile-time constant
- Prevent full table scans from wiping out the InnoDB buffer cache
- Make prefetching smarter
- Get feedback from Dimitri, Domas, Mikael and Percona
- Use prefetch with MRR/BKA to get parallel IO in InnoDB
- Investigate larger doublewrite buffer to allow for more concurrent IOs
- Make Innodb work with a 4kb page size
- Make trx_purge() faster when called by the main background thread
- Use crc32 for Innodb page checksums with hardware support or otherwise make checksum faster.
- Reduce the per-page overhead for sync objects
- Repeat
- Add my.cnf options to disable InnoDB prefetch reads
- Put more output in SHOW INNODB STATUS and SHOW STATUS
- Reduce the overhead from buf_flush_free_margin()
- Change background IO threads to use available IO capacity
- Use more IO to merge insert buffer records when the insert buffer is full
- Fix mutex contention for the HEAP engine
- Fix mutex contention for the MyISAM engine
- Fix mutex contention for the query cache
- Give priority (CPU, disk) to the replication SQL thread to minimize replication delay.
- Push changes for --oltp-secondary-index to public sysbench branch
- Add support to sysbench fileio for transaction log and doublewrite buffer IO patterns
Posts
Mark, one theoretical item I've thought about after reading the code, but never really proven to be a problem -- the way the log buffer is written and flushed. I wonder if the fact that flushing blocks log buffer writes, and then moves a bunch of data around (the remaining unflushed stuff), is a problem.
ReplyDeleteI have not seen that to be a problem (yet). Ben has changes in the v3 patch that reduce mutex contention between log_sys->mutex and buffer_pool->mutex.
ReplyDeleteMark,
ReplyDeleteFirst, let me say how much we at Innobase appreciate your interest, support and investment in InnoDB. Please don't hesitate to contact us as you work on these things.
While we can't a priori agree to incorporate any specific patch that anyone may come up with, we are definitely interested in looking at all potential contributions that can improve performance or functionality.
We have to consider a number of things, of course, including making sure the patch applies to the InnoDB Plugin (not just the builtin InnoDB), and we do need to make patches portable (because users run on platforms other than Linux). We are more than willing to do that work, of course, but we have to apply our resources selectively.
One thing that would help a lot is if we could see a demonstration of proposed patches in isolation, just one at a time. When we see a group of patches all tested at once, we have no way to tell whether any given patch makes a significant contribution or not. We would look for a "scientific method", where only ONE thing is changed in a controlled experiment.
In performing these tests, it is helpful if the baseline is relevant too ... when we see results of some workload running on an unpatched 5.0 compared to a heavily patched 5.1, it's really hard to tell the difference any given patch makes.
We also need to see concrete evidence of performance gains for individual patches, not just a rational / theoretical argument about how things are supposed to work. I know you know the subtleties here and recognize that sometimes a change may have unexpected (or unwanted) effects.
It also helps to know which workloads are improved, which may not be affected ... and which might be negatively impacted.
Finally, because multiple patches could address similar or related areas, a disciplined approach is to investigate patches in order of the magnitude of their improvement. We would like to incorporate a patch that provides the most significant benefit before we consider others. It may turn out that the contribution of one patch means that another patch is not useful or necessary.
All that said, again thank you, and keep up the good work!
One more item: make auto_increment persistent, so when we restart, we don´t lose the latest value.
ReplyDelete(I know it´s not "performance" related, but the excuse I´ve been listening for the last years, it´s that this is a performance issue)
Horace,
ReplyDeleteThat is a great request, but I won't do it. Maybe Percona or Innobase will.