I published some of the code I use for performance testing at code.google.com:

• ibench.py - this implements the insert benchmark from Tokutek. It optionally runs concurrent query threads to simulate the load on a busy replica. This workload can generate a lot of IO for stress testing or to understand the limits of a storage engine.
  
• mstat.py - this collects performance data from iostat, vmstat and SHOW STATUS. A sample config file, mstat.ds, is also provided. It can be used to understand the load on a server during benchmarks or any other time you need to explain performance.
  

I posted a patch for sysbench here. This adds a new flag, --oltp-secondary-index, that can be used for the OLTP tests. When this flag is set another column and secondary index are added to the sbtest table and half of the queries use the secondary index. The default behavior is used when it is not set, and all query predicates use the primary key on sbtest.

Feedback is welcome. I need to determine whether I should submit the patch to the sysbench maintainer. sysbench is a very nice tool for benchmarking. There are more complex tests, but if a system doesn't do well on sysbench, it probably will have problems on everything else. So it is a great place to start.