use*_*247 6 mysql innodb my.cnf mysql-5.5
早上好,
我目前面临一个问题,即 CPU 使用率一直上升,直到服务器崩溃。
我已经尝试了在这个论坛和 Percona 上提出的多个最佳实践,已经重新安装了两次服务器。似乎没有任何效果。
该应用程序是一个小脚本,它以非常密集的方式对范围索引执行 SELECT、对主键执行更新以及对 UNIQUE 键执行批量插入。
99.9% 的查询在单个 InnoDB 表上运行。
以下是 mysqltuner.pl 输出(根本没有帮助我):
[--] Reads / Writes: 12% / 88%
[--] Total buffers: 4.2G global + 5.0M per thread (100 max threads)
[OK] Maximum possible memory usage: 4.7G (59% of installed RAM)
[OK] Slow queries: 0% (1/1M)
[OK] Highest usage of available connections: 42% (42/100)
[OK] Key buffer size / total MyISAM indexes: 128.0M/29.3M
[!!] Query cache is disabled
[OK] Sorts requiring temporary tables: 0% (0 temp sorts / 2 sorts)
[OK] Temporary tables created on disk: 19% (63 on disk / 327 total)
[OK] Thread cache hit rate: 58% (42 created / 102 connections)
[OK] Table cache hit rate: 38% (126 open / 325 opened)
[OK] Open file limit used: 3% (68/2K)
[OK] Table locks acquired immediately: 96% (1M immediate / 1M locks)
[OK] InnoDB buffer pool / data size: 4.0G/358.1M
[OK] InnoDB log waits: 0
My.cnf 文件已根据我的知识以及我所看到的专业人士使用或建议的内容进行了优化:
# general tuning
# family of join_buffer_size ,sort_buffer_size, read_buffer_size, read_rnd_buffer_size
sort_buffer_size = 4M
read_rnd_buffer_size = 512K
table_cache = 1024
key_buffer_size = 128M
#show global status like 'Max_used_connections'
max_connections = 100
#determine how many thread MySQL keep in memory to handle new conn
#100 - ((Threads_created/Connections)*100)
thread_cache_size = 50
#max size of what is being cached see SHOW GLOBAL STATUS LIKE 'Qc%'
#query_cache_limit = 2M
query_cache_size = 0
query_cache_type = 0
#innodb tuning
innodb_buffer_pool_size = 4G
innodb_buffer_pool_instances = 2
innodb_log_file_size = 1G
innodb_log_buffer_size = 16MB
innodb_flush_log_at_trx_commit = 2
innodb_file_per_table
innodb_flush_method = O_DIRECT
innodb_read_io_threads = 64
innodb_write_io_threads = 64
innodb_thread_concurrency = 0
Run Code Online (Sandbox Code Playgroud)
到目前为止,我已经尝试:
1) 更改query_cache_size
签入对 CPU 使用率有任何影响。一直无法从中看到任何好处,所以我禁用了它。
2)thread_cache
为了获得更好的缓存命中率而改变。没有变化,Threads_created
所以我决定让它在 50。
3) innodb_buffer_pool_size
= 4G(总共 8G)
运行 rolando 的查询:
SELECT CEILING(Total_InnoDB_Bytes*1.6/POWER(1024,3)) RIBPS FROM
(SELECT SUM(data_length+index_length) Total_InnoDB_Bytes FROM
information_schema.tables WHERE engine='InnoDB') A;
Run Code Online (Sandbox Code Playgroud)
在 mycase 中返回 2G。所以我决定有 2 个带有innodb_read_io_threads = 64
和 的缓冲池实例innodb_write_io_threads = 64
。
对此一无所获。
4) 调整innodb_log_file_size
到池的 25% 并将 trx_commit 设为 2。
还是没有改善。
正常操作模式下的 innodb 状态输出是(尝试以可读的方式粘贴它)。
mysql> show engine innodb status\G
*************************** 1. row ***************************
Type: InnoDB
Name:
Status:
=====================================
140420 15:41:43 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 38 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 62 1_second, 61 sleeps, 6 10_second, 1 background, 1 flush
srv_master_thread log flush and writes: 62
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 722, signal count 2177
Mutex spin waits 351436, rounds 271903, OS waits 201
RW-shared spins 1069, rounds 12114, OS waits 242
RW-excl spins 479, rounds 10793, OS waits 234
Spin rounds per wait: 0.77 mutex, 11.33 RW-shared, 22.53 RW-excl
------------
TRANSACTIONS
------------
Trx id counter A6BFB
Purge done for trx's n:o < A373C undo n:o < 0
History list length 8780
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION A6BB0, not started
MySQL thread id 80, OS thread handle 0x7f4fb7c69700, query id 149238 localhost data
---TRANSACTION A6BD2, not started
MySQL thread id 86, OS thread handle 0x7f4fb7ae3700, query id 149642 localhost data
---TRANSACTION A6BC9, not started
MySQL thread id 78, OS thread handle 0x7f4fb7ceb700, query id 149544 localhost data
---TRANSACTION A6BB8, not started
MySQL thread id 70, OS thread handle 0x7f4fb7ef3700, query id 149353 localhost data
---TRANSACTION A6BD6, not started
MySQL thread id 73, OS thread handle 0x7f4fb7e30700, query id 149674 localhost data
---TRANSACTION A6BBB, not started
MySQL thread id 79, OS thread handle 0x7f4fb7caa700, query id 149298 localhost data
---TRANSACTION A6BC0, not started
MySQL thread id 84, OS thread handle 0x7f4fb7b65700, query id 149379 localhost data
---TRANSACTION A6BCF, not started
MySQL thread id 63, OS thread handle 0x7f4fb80ba700, query id 149574 localhost data
---TRANSACTION A6BCC, not started
MySQL thread id 61, OS thread handle 0x7f5112b16700, query id 149616 localhost data
---TRANSACTION A6BC6, not started
MySQL thread id 58, OS thread handle 0x7f5112bd9700, query id 149498 localhost data
---TRANSACTION 0, not started
MySQL thread id 47, OS thread handle 0x7f5112c1a700, query id 149675 localhost savas
show engine innodb status
---TRANSACTION A6BF9, ACTIVE 0 sec
mysql tables in use 1, locked 0
MySQL thread id 62, OS thread handle 0x7f5112ad5700, query id 149649 localhost data Sending data
SELECT id FROM tracker WHERE domain_id=2 AND visited='Y' AND stop_visit='N' AND NOW()>=addtime(updated,sec_to_time('43200')) limit 100
Trx read view will not see trx with id >= A6BFA, sees < A3706
---TRANSACTION A6BF3, ACTIVE 0 sec
2 lock struct(s), heap size 376, 1 row lock(s)
MySQL thread id 81, OS thread handle 0x7f4fb7c28700, query id 149612 localhost data
Trx read view will not see trx with id >= A6BF4, sees < A3706
---TRANSACTION A6BEE, ACTIVE 0 sec
2 lock struct(s), heap size 376, 1 row lock(s)
MySQL thread id 75, OS thread handle 0x7f4fb7dae700, query id 149515 localhost data
Trx read view will not see trx with id >= A6BEF, sees < A3706
---TRANSACTION A6BEB, ACTIVE 0 sec
2 lock struct(s), heap size 376, 1 row lock(s)
MySQL thread id 82, OS thread handle 0x7f4fb7be7700, query id 149465 localhost data
Trx read view will not see trx with id >= A6BEC, sees < A3706
---TRANSACTION A6BE8, ACTIVE 0 sec
2 lock struct(s), heap size 376, 1 row lock(s)
MySQL thread id 60, OS thread handle 0x7f5112b57700, query id 149449 localhost data
Trx read view will not see trx with id >= A6BE9, sees < A3706
---TRANSACTION A6BE5, ACTIVE 0 sec
2 lock struct(s), heap size 376, 1 row lock(s)
MySQL thread id 59, OS thread handle 0x7f5112b98700, query id 149419 localhost data
Trx read view will not see trx with id >= A6BE6, sees < A3706
---TRANSACTION A6BE4, ACTIVE 0 sec
2 lock struct(s), heap size 376, 1 row lock(s)
MySQL thread id 83, OS thread handle 0x7f4fb7ba6700, query id 149420 localhost data
Trx read view will not see trx with id >= A6BE5, sees < A3706
---TRANSACTION A6BDC, ACTIVE 0 sec
2 lock struct(s), heap size 376, 1 row lock(s)
MySQL thread id 71, OS thread handle 0x7f4fb7eb2700, query id 149327 localhost data
Trx read view will not see trx with id >= A6BDD, sees < A3706
---TRANSACTION A6BD9, ACTIVE 0 sec
2 lock struct(s), heap size 376, 1 row lock(s)
MySQL thread id 65, OS thread handle 0x7f4fb8038700, query id 149312 localhost data
Trx read view will not see trx with id >= A6BDA, sees < A3706
---TRANSACTION A6BB7, ACTIVE 0 sec
2 lock struct(s), heap size 376, 1 row lock(s)
MySQL thread id 68, OS thread handle 0x7f4fb7f75700, query id 149058 localhost data
Trx read view will not see trx with id >= A6BB8, sees < A3706
---TRANSACTION A3727, ACTIVE 63 sec
MySQL thread id 96, OS thread handle 0x7f4fb7859700, query id 48112 localhost data
Trx read view will not see trx with id >= A3728, sees < A3700
---TRANSACTION A3726, ACTIVE 63 sec
MySQL thread id 97, OS thread handle 0x7f4fb7818700, query id 47827 localhost data
Trx read view will not see trx with id >= A3727, sees < A3700
---TRANSACTION A3725, ACTIVE 63 sec
MySQL thread id 95, OS thread handle 0x7f4fb789a700, query id 47424 localhost data
Trx read view will not see trx with id >= A3726, sees < A3700
---TRANSACTION A3724, ACTIVE 63 sec
MySQL thread id 85, OS thread handle 0x7f4fb7b24700, query id 47707 localhost data
Trx read view will not see trx with id >= A3725, sees < A3700
---TRANSACTION A3723, ACTIVE 63 sec
MySQL thread id 89, OS thread handle 0x7f4fb7a20700, query id 47625 localhost data
Trx read view will not see trx with id >= A3724, sees < A3700
---TRANSACTION A3722, ACTIVE 63 sec
MySQL thread id 92, OS thread handle 0x7f4fb795d700, query id 48674 localhost data
Trx read view will not see trx with id >= A3723, sees < A3700
---TRANSACTION A3721, ACTIVE 63 sec
MySQL thread id 93, OS thread handle 0x7f4fb791c700, query id 47449 localhost data
Trx read view will not see trx with id >= A3722, sees < A3700
---TRANSACTION A3720, ACTIVE 63 sec
MySQL thread id 90, OS thread handle 0x7f4fb79df700, query id 48764 localhost data
Trx read view will not see trx with id >= A3721, sees < A3700
---TRANSACTION A371F, ACTIVE 63 sec
MySQL thread id 94, OS thread handle 0x7f4fb78db700, query id 48320 localhost data
Trx read view will not see trx with id >= A3720, sees < A3700
---TRANSACTION A371E, ACTIVE 63 sec
MySQL thread id 91, OS thread handle 0x7f4fb799e700, query id 48109 localhost data
Trx read view will not see trx with id >= A371F, sees < A3700
---TRANSACTION A371D, ACTIVE 63 sec
MySQL thread id 76, OS thread handle 0x7f4fb7d6d700, query id 48220 localhost data
Trx read view will not see trx with id >= A371E, sees < A3700
---TRANSACTION A3718, ACTIVE 63 sec
MySQL thread id 88, OS thread handle 0x7f4fb7a61700, query id 48238 localhost data
Trx read view will not see trx with id >= A3719, sees < A3700
---TRANSACTION A3716, ACTIVE 63 sec
MySQL thread id 87, OS thread handle 0x7f4fb7aa2700, query id 48036 localhost data
Trx read view will not see trx with id >= A3717, sees < A3700
---TRANSACTION A3711, ACTIVE 63 sec
MySQL thread id 77, OS thread handle 0x7f4fb7d2c700, query id 48232 localhost data
Trx read view will not see trx with id >= A3712, sees < A3700
---TRANSACTION A370E, ACTIVE 63 sec
MySQL thread id 72, OS thread handle 0x7f4fb7e71700, query id 47602 localhost data
Trx read view will not see trx with id >= A370F, sees < A3700
---TRANSACTION A370C, ACTIVE 63 sec
MySQL thread id 74, OS thread handle 0x7f4fb7def700, query id 48101 localhost data
Trx read view will not see trx with id >= A370D, sees < A3700
---TRANSACTION A370B, ACTIVE 63 sec
MySQL thread id 69, OS thread handle 0x7f4fb7f34700, query id 48102 localhost data
Trx read view will not see trx with id >= A370C, sees < A3700
---TRANSACTION A3708, ACTIVE 63 sec
MySQL thread id 67, OS thread handle 0x7f4fb7fb6700, query id 48748 localhost data
Trx read view will not see trx with id >= A3709, sees < A3700
---TRANSACTION A3707, ACTIVE 63 sec
MySQL thread id 66, OS thread handle 0x7f4fb7ff7700, query id 48263 localhost data
Trx read view will not see trx with id >= A3708, sees < A3700
---TRANSACTION A3706, ACTIVE 63 sec
MySQL thread id 64, OS thread handle 0x7f4fb8079700, query id 48314 localhost data
Trx read view will not see trx with id >= A3707, sees < A3700
--------
FILE I/O
--------
I/O thread 0 state: waiting for completed aio requests (insert buffer thread)
I/O thread 1 state: waiting for completed aio requests (log thread)
I/O thread 2 state: waiting for completed aio requests (read thread)
I/O thread 3 state: waiting for completed aio requests (read thread)
I/O thread 4 state: waiting for completed aio requests (read thread)
I/O thread 5 state: waiting for completed aio requests (read thread)
I/O thread 6 state: waiting for completed aio requests (read thread)
I/O thread 7 state: waiting for completed aio requests (read thread)
I/O thread 8 state: waiting for completed aio requests (read thread)
I/O thread 9 state: waiting for completed aio requests (read thread)
I/O thread 10 state: waiting for completed aio requests (read thread)
I/O thread 11 state: waiting for completed aio requests (read thread)
I/O thread 12 state: waiting for completed aio requests (read thread)
I/O thread 13 state: waiting for completed aio requests (read thread)
I/O thread 14 state: waiting for completed aio requests (read thread)
I/O thread 15 state: waiting for completed aio requests (read thread)
I/O thread 16 state: waiting for completed aio requests (read thread)
I/O thread 17 state: waiting for completed aio requests (read thread)
I/O thread 18 state: waiting for completed aio requests (read thread)
I/O thread 19 state: waiting for completed aio requests (read thread)
I/O thread 20 state: waiting for completed aio requests (read thread)
I/O thread 21 state: waiting for completed aio requests (read thread)
I/O thread 22 state: waiting for completed aio requests (read thread)
I/O thread 23 state: waiting for completed aio requests (read thread)
I/O thread 24 state: waiting for completed aio requests (read thread)
I/O thread 25 state: waiting for completed aio requests (read thread)
I/O thread 26 state: waiting for completed aio requests (read thread)
I/O thread 27 state: waiting for completed aio requests (read thread)
I/O thread 28 state: waiting for completed aio requests (read thread)
I/O thread 29 state: waiting for completed aio requests (read thread)
I/O thread 30 state: waiting for completed aio requests (read thread)
I/O thread 31 state: waiting for completed aio requests (read thread)
I/O thread 32 state: waiting for completed aio requests (read thread)
I/O thread 33 state: waiting for completed aio requests (read thread)
I/O thread 34 state: waiting for completed aio requests (read thread)
I/O thread 35 state: waiting for completed aio requests (read thread)
I/O thread 36 state: waiting for completed aio requests (read thread)
I/O thread 37 state: waiting for completed aio requests (read thread)
I/O thread 38 state: waiting for completed aio requests (read thread)
I/O thread 39 state: waiting for completed aio requests (read thread)
I/O thread 40 state: waiting for completed aio requests (read thread)
I/O thread 41 state: waiting for completed aio requests (read thread)
I/O thread 42 state: waiting for completed aio requests (read thread)
I/O thread 43 state: waiting for completed aio requests (read thread)
I/O thread 44 state: waiting for completed aio requests (read thread)
I/O thread 45 state: waiting for completed aio requests (read thread)
I/O thread 46 state: waiting for completed aio requests (read thread)
I/O thread 47 state: waiting for completed aio requests (read thread)
I/O thread 48 state: waiting for completed aio requests (read thread)
I/O thread 49 state: waiting for completed aio requests (read thread)
I/O thread 50 state: waiting for completed aio requests (read thread)
I/O thread 51 state: waiting for completed aio requests (read thread)
I/O thread 52 state: waiting for completed aio requests (read thread)
I/O thread 53 state: waiting for completed aio requests (read thread)
I/O thread 54 state: waiting for completed aio requests (read thread)
I/O thread 55 state: waiting for completed aio requests (read thread)
I/O thread 56 state: waiting for completed aio requests (read thread)
I/O thread 57 state: waiting for completed aio requests (read thread)
I/O thread 58 state: waiting for completed aio requests (read thread)
I/O thread 59 state: waiting for completed aio requests (read thread)
I/O thread 60 state: waiting for completed aio requests (read thread)
I/O thread 61 state: waiting for completed aio requests (read thread)
I/O thread 62 state: waiting for completed aio requests (read thread)
I/O thread 63 state: waiting for completed aio requests (read thread)
I/O thread 64 state: waiting for completed aio requests (read thread)
I/O thread 65 state: waiting for completed aio requests (read thread)
I/O thread 66 state: waiting for completed aio requests (write thread)
I/O thread 67 state: waiting for completed aio requests (write thread)
I/O thread 68 state: waiting for completed aio requests (write thread)
I/O thread 69 state: waiting for completed aio requests (write thread)
I/O thread 70 state: waiting for completed aio requests (write thread)
I/O thread 71 state: waiting for completed aio requests (write thread)
I/O thread 72 state: waiting for completed aio requests (write thread)
I/O thread 73 state: waiting for completed aio requests (write thread)
I/O thread 74 state: waiting for completed aio requests (write thread)
I/O thread 75 state: waiting for completed aio requests (write thread)
I/O thread 76 state: waiting for completed aio requests (write thread)
I/O thread 77 state: waiting for completed aio requests (write thread)
I/O thread 78 state: waiting for completed aio requests (write thread)
I/O thread 79 state: waiting for completed aio requests (write thread)
I/O thread 80 state: waiting for completed aio requests (write thread)
I/O thread 81 state: waiting for completed aio requests (write thread)
I/O thread 82 state: waiting for completed aio requests (write thread)
I/O thread 83 state: waiting for completed aio requests (write thread)
I/O thread 84 state: waiting for completed aio requests (write thread)
I/O thread 85 state: waiting for completed aio requests (write thread)
I/O thread 86 state: waiting for completed aio requests (write thread)
I/O thread 87 state: waiting for completed aio requests (write thread)
I/O thread 88 state: waiting for completed aio requests (write thread)
I/O thread 89 state: waiting for completed aio requests (write thread)
I/O thread 90 state: waiting for completed aio requests (write thread)
I/O thread 91 state: waiting for completed aio requests (write thread)
I/O thread 92 state: waiting for completed aio requests (write thread)
I/O thread 93 state: waiting for completed aio requests (write thread)
I/O thread 94 state: waiting for completed aio requests (write thread)
I/O thread 95 state: waiting for completed aio requests (write thread)
I/O thread 96 state: waiting for completed aio requests (write thread)
I/O thread 97 state: waiting for completed aio requests (write thread)
I/O thread 98 state: waiting for completed aio requests (write thread)
I/O thread 99 state: waiting for completed aio requests (write thread)
I/O thread 100 state: waiting for completed aio requests (write thread)
I/O thread 101 state: waiting for completed aio requests (write thread)
I/O thread 102 state: waiting for completed aio requests (write thread)
I/O thread 103 state: waiting for completed aio requests (write thread)
I/O thread 104 state: waiting for completed aio requests (write thread)
I/O thread 105 state: waiting for completed aio requests (write thread)
I/O thread 106 state: waiting for completed aio requests (write thread)
I/O thread 107 state: waiting for completed aio requests (write thread)
I/O thread 108 state: waiting for completed aio requests (write thread)
I/O thread 109 state: waiting for completed aio requests (write thread)
I/O thread 110 state: waiting for completed aio requests (write thread)
I/O thread 111 state: waiting for completed aio requests (write thread)
I/O thread 112 state: waiting for completed aio requests (write thread)
I/O thread 113 state: waiting for completed aio requests (write thread)
I/O thread 114 state: waiting for completed aio requests (write thread)
I/O thread 115 state: waiting for completed aio requests (write thread)
I/O thread 116 state: waiting for completed aio requests (write thread)
I/O thread 117 state: waiting for completed aio requests (write thread)
I/O thread 118 state: waiting for completed aio reque
我将解释我是如何解决困扰我数周的性能问题的。随意批评,结果只会对有需要的人更好。
首先,我阅读了“高性能 MySQL”一书。这是一本很棒的书,因为它们从一开始就揭示了分析方法。基本上,当您阅读完前 2-3 章后,您就拥有了足够的工具来开始挖掘。我不知道大多数分析方法,但我学到了很多。
我决定通过设置从 query_log 开始挖掘query_log_time=0
。然后我选择了导致麻烦的查询。事实上,因为所有查询都被索引,所以只有一种选择是可能的,但是这个特殊的选择是在一个大表上工作的。经历了这里的一切。
目标表是:
+------------+---------------+------+-----+-------------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+---------------+------+-----+-------------------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| domain_id | int(11) | NO | MUL | NULL | |
| url | varchar(250) | NO | UNI | 0 | |
| visited | enum('Y','N') | NO | | N | |
| updated | timestamp | NO | | CURRENT_TIMESTAMP | |
| hits | smallint(6) | NO | | 0 | |
| stop_visit | enum('Y','N') | NO | | N | |
+------------+---------------+------+-----+-------------------+----------------+
Run Code Online (Sandbox Code Playgroud)
可疑的查询是:
SELECT id FROM tracker WHERE updated<=NOW() and domain_id=2 and stop_visit='N' LIMIT 1;
Run Code Online (Sandbox Code Playgroud)
(updated,domain_id,stop_visit) 上有一个 BTREE 索引,所以解释计划没有给出任何内容。然后我尝试像这样查询分析:
SET profiling=1;
SHOW profiles;
SET @query_id=1
SELECT STATE, SUM(DURATION) AS Total_R,
ROUND(
100 * SUM(DURATION) /
(SELECT SUM(DURATION)
FROM INFORMATION_SCHEMA.PROFILING
WHERE QUERY_ID = @query_id
), 2) AS Pct_R,
COUNT(*) AS Calls,
SUM(DURATION) / COUNT(*) AS "R/Call"
FROM INFORMATION_SCHEMA.PROFILING
WHERE QUERY_ID = @query_id
GROUP BY STATE
ORDER BY Total_R DESC;
Run Code Online (Sandbox Code Playgroud)
没有观察到任何错误,并且查询正在以低于 0.001 的时间执行。没有临时表,使用索引......一切看起来都不错,但显然有一个我知道它存在的潜在问题。
SHOW STATUS
并FLUSH STATUS
展示了一些非常奇怪的东西:
FLUSH status;
SELECT id FROM tracker WHERE updated<=NOW() and domain_id=2 and stop_visit='N' LIMIT 1;
show status where variable_name like 'Created%' or variable_name like 'Handler%';
+----------------------------+-------+
| Variable_name | Value |
+----------------------------+-------+
| Created_tmp_disk_tables | 0 |
| Created_tmp_files | 6 |
| Created_tmp_tables | 0 |
| Handler_commit | 0 |
| Handler_delete | 0 |
| Handler_discover | 0 |
| Handler_prepare | 0 |
| Handler_read_first | 0 |
| Handler_read_key | 0 |
| Handler_read_last | 0 |
| Handler_read_next | 44687 |
| Handler_read_prev | 0 |
| Handler_read_rnd | 0 |
| Handler_read_rnd_next | 0 |
| Handler_rollback | 0 |
| Handler_savepoint | 0 |
| Handler_savepoint_rollback | 0 |
| Handler_update | 0 |
| Handler_write | 0 |
+----------------------------+-------+
Run Code Online (Sandbox Code Playgroud)
现在我想为什么当索引被定制为选择性时 Handler_read_next 这么高(或者我是这么认为的)。好吧,事实并非如此。更重要的是,这个问题我仍然没有答案。经过一系列随机尝试后,我发现 ENUM 列(stop_visit)导致了这个非常高的 Handler_read_next 值。将数据类型更改为 tinyint [0,1] 或 char(1) ['Y','N'] 对我没有任何回报。
在这一点上,我决定打破一切,用所学的知识从头开始。这导致对初始表的以下更改:
+-----------+---------------------+------+-----+-------------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+---------------------+------+-----+-------------------+----------------+
| id | bigint(20) unsigned | NO | PRI | NULL | auto_increment |
| url_crc | int(10) unsigned | NO | MUL | NULL | |
| updated | timestamp | NO | MUL | CURRENT_TIMESTAMP | |
| domain_id | tinyint(3) unsigned | NO | | NULL | |
| owner | int(10) unsigned | NO | MUL | 0 | |
| url | varchar(255) | NO | | NULL | |
| hits | tinyint(3) unsigned | NO | | 0 | |
+-----------+---------------------+------+-----+-------------------+----------------+
Run Code Online (Sandbox Code Playgroud)
之前的多列索引变成了单列索引ALTER table tracker add index idx_updated (updated)
。
结果是一个没有执行任何 Handler_read_next 操作的查询。相反,它优雅地将 Handler_read_key 和 Handler_read_first 增加了 1。
使用本书中描述的用于访问 VARCHAR 数据类型的方法解决了另一个模糊的问题。我以前在 url 列上有这个 UNIQUE 索引,我曾经在这个大表上插入 IGNORE 认为信任 UNIQUE 约束是要走的路。所以我想,如果 Mysql 必须遍历所有行来插入这个新 url,那会很痛苦。原因是大约 40 INSERT IGNORE 每 0.001(第一个查询完成后)平均 50 个并发客户端。
所以我通过使用 crc32 整数索引改变了这种行为。然后像这样阅读:
SELECT * from tracker WHERE url_crc=crc32('http://google.com') and url='http://google.com'
Run Code Online (Sandbox Code Playgroud)
每 0.001 仍然有 40 个 SELECT,但至少我可以轻松地分析它们。所以我决定暂时坚持这个。
在完成所有这些更改之后,验证所有工作的时间到了。
再次从这本伟大的书中 3 个简单的方法:
> while true; do sleep 1; mysql -e 'SHOW PROCESSLIST\G' | grep State: | sort | uniq -c | sort -rn; done
> mysqladmin ext -i1 | awk 'BEGIN{printf "%5s %5s %5s %5s %5s %5s %5s\n","QPS","Thcon","Thrun","Opentb","tmp_tbl","FreePages","Bytessent"}/Queries/{q=$4-qp;qp=$4}/Threads_connected/{tc=$4}/Created_tmp_disk_tables/{tmp=$4}/Opened_tables/{ot=$4}/Innodb_buffer_pool_pages_free/{pf=$4}/Bytes_sent/{bs=$4}/Threads_running/{printf "%5d %5d %5d %5d %5d %10d %10d\n",q,tc,$4,ot,tmp,pf,bs}'
> awk '/^# Time:/{print $3, $4, c;c=0}/^# User/{c++}' mysql-slow.log
Run Code Online (Sandbox Code Playgroud)
第一个命令将有助于发现那些不正常的状态。这也是更高级分析的开始,但我不会在这里介绍这些方面。该命令帮助我发现查询缓存弊大于利。
第二个命令用于检查服务器的状态并查看它是如何发展的。只需查看输出即可快速发现问题:
QPS Th_con Th_run Opened_files tmp_disk free_pages bytes_sent
466 22 7 276 54 120658 30110619
401 22 6 276 54 120657 30145343
405 22 9 276 54 120657 30179509
=>85 22 5 276 54 120658 30196291
Run Code Online (Sandbox Code Playgroud)
QPS 突然下降到 85 是警告出现问题。就我而言,query_log 显示了大量更新。这是由于缓存表没有以一种很酷的方式刷新 MySQL 造成的。
通过所有这些改进,我设法对我的路由器进行了 DoS kernel: [539677.201824] nf_conntrack: table full, dropping packet.
。那是第一次:p
所以我认为这就是感谢伟大著作《高性能 MySQL》的作者和社区的所有相关帖子的时间。