If you find yourself with a MySQL query that's taking forever to complete, the suspects at the top of your troubleshooting list usually relate to the approach MySQL is choosing to index the table being searched. Discovering the cause of the query slow-down is only the beginning: the "key" is to ensure the system is selecting the optimal index option.
MySQL's index_merge function is intended to allow some queries that contain WHERE clauses and single-column indexes to use multiple indexes. The goal is to speed up queries by searching only specific indexes of tables rather than the entire table via the default PRIMARY KEY.
In a December 2012 post on the MySQL Performance Blog, Ernie Souhrada uses the example of the query “SELECT foo FROM bar WHERE indexed_colA = X OR indexed_colB = Y” that applies the index merge union algorithm to scan the "indexed_colA" and "indexed_colB" columns simultaneously. Then it performs a set-theoretic union of the two result sets. (Using "AND" in place of "OR" generates the set-theoretic intersection of the result sets.)
Index_merge sometimes clobbers performance, however. Souhrada gives the example of an index merge run on a table that had 4.5 million rows. The EXPLAIN that was run on the SELECT indicated three different indexes were being used to search about 8,100 of the table's 4.5 million rows.
A query executing multiple indexes appears to be searching only a fraction of the table's rows. Source: MySQL Performance Blog
In fact, the query was taking 3.3 seconds, far longer than expected. The specific conditions in this query were causing MySQL to search 8.5 million rows rather than the expected 8,100. The query was reading the index entries for all columns in the WHERE clause that were included in the merge operation. It then performed a set intersection on the results.
The solution in this case was to use index hints, which are described in the MySQL Reference Manual. Rewriting the query to focus on user_type reduces processing time from 3.3 seconds to a millisecond.
Using an index hint to focus the query on a single index removed the performance bottleneck. Source: MySQL Performance Blog
While an index hint was the best solution in this case, in other situations an index hint can cause more problems than it solves. You simply can't anticipate future changes to the database and to MySQL itself in subsequent versions, any of which could transform the index hint into a time bomb waiting to explode your database.
Many ways to ensure MySQL queries read your preferred index
As the above example shows, finding the source of the slow index processing is only the beginning of your troubleshooting. There are almost always multiple potential solutions to a query-index problem. On his OpenArk.org blog, Shlomi Noach lists seven alternative approaches to speeding up MySQL index queries.
A table with about 10 million rows ran a query indexed to the "type" key, which should limit queries to scanning about 110 rows, filtered "using where" and sorted "using filesort". Instead, the query was using PRIMARY KEY to search all 10 million rows, filtered "using where."
For some reason, MySQL was ignoring the plan it identified as most efficient. The first options are to rebuild the table or to run ANALYZE TABLE to update the index statistics, which takes less time than rebuilding and may help generate better query plans. The bulldozer approach is to use FORCE INDEX, or alternatively to limit MySQL to using only the index you specify.
One option to address slow MySQL queries is to state explicitly the index to be searched. Source: OpenArk.org
Alternatively, you could instruct MySQL to ignore the PRIMARY KEY:
An alternative for speeding up queries is to specify that MySQL should ignore the PRIMARY KEY. Source: OpenArk.org
One downside to these approaches is that they are not standard SQL. Moving some logic to the application would be a possibility, but this solution is rarely quick or simple. Versions of SQL that support the ORDER BY function could specifically negate use of PRIMARY KEY. You can realize the same result by "tricking" MySQL into believing the PRIMARY KEY is not optimal via use of "ORDER BY id". This makes the second sorting column redundant, so MySQL concludes on its own that use of the "type" index is best.
DBAs can spend a considerable amount of their workday puzzling over these types of MySQL performance issues. What few admins look forward to is writing backup scripts, scheduling backups, and ensuring their databases are fully recoverable. The Morpheus Virtual Appliance is your new solution for database provisioning and managing on private, public, and hybrid clouds. Start your free trial now!