为什么MySQL InnoDB在全表扫描时比MyISAM慢得多？

Question

编辑

OP在下面的回答中分析PostgreSQL时承认了一个错误.我正在更新此问题以反映MyISAM和InnoDB之间的比较.

你好,

我针对MySQL InnoDB,MyISAM和PostgreSQL进行了测试,看看这些引擎在执行全表扫描时的表现如何,以便了解对于我们不可避免地需要发生这种情况的响应时间.

测试是在Intel Core 2 Quad Q6600 @ 2.4Ghz w/4GB RAM和7200 RPM HD(16MB缓存)上进行的.

MySQL版本为5.0.67-community-nt-log 32位,PGSQL版本为8.4.

我写了一个小脚本,在4列表中生成500万行数据.这些是MySQL和PGSQL中使用的create table语句:

- InnoDB

CREATE TABLE sample_innodb (
id integer unsigned not null,
vc1 varchar(200) not null,
vc2 varchar(200) not null,
vc3 varchar(200) not null
) ENGINE=InnoDB;

- MyISAM

CREATE TABLE sample_isam (
id integer unsigned not null,
vc1 varchar(200) not null,
vc2 varchar(200) not null,
vc3 varchar(200) not null
) ENGINE=MyISAM;

- PostgreSQL

create table sample_pgsql (
id integer not null,
vc1 varchar(200) not null,
vc2 varchar(200) not null,
vc3 varchar(200) not null
);

这是我用来为这些表生成数据的脚本:

var chars = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXTZabcdefghiklmnopqrstuvwxyz'.split('');

function randomString(length) {
 var str = '';
 for (var i = 0; i < length; i++) {
    str += chars[Math.floor(Math.random() * chars.length)];
 }

   return str;
}

function genrow(idv, vcv1, vcv2, vcv3) {
 return idv + "," + vcv1 + "," + vcv2 + "," + vcv3;
}

function gentable(numrows) {
 for (var i = 0; i < numrows; i++) {
    var row = 
        genrow(i,
               randomString(10),
               randomString(20),
               randomString(30));

    WScript.Echo(row);
  }
}

gentable(5000000);

我使用以下命令在Windows上运行此脚本:

cscript.exe /nologo test.js > data.csv

您可以使用以下命令将此数据加载到MySQL中:

LOAD DATA LOCAL INFILE 'data.csv'
INTO TABLE sample_innodb
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
(id, vc1, vc2, vc3);

LOAD DATA LOCAL INFILE 'data.csv'
INTO TABLE sample_isam
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
(id, vc1, vc2, vc3);

您可以使用以下命令将数据加载到PGSQL中:

copy sample_pgsql (id, vc1, vc2, vc3) from 'data.csv' with delimiter ','

我使用此查询来尝试强制最坏情况表扫描方案的时间:

MySQL的

select count(*) from [table] 
where vc1 like '%blah0%' and vc2 like '%blah1%' and vc3 like '%blah2%';

PostgreSQL的

select count(*) from [table] 
where vc1 ilike '%blah0%' and vc2 ilike '%blah1%' and vc3 ilike '%blah2%';

我多次运行此查询以获得平均完成时间,省去第一次运行以使所有内容都在内存中启动.

结果如下:

• InnoDB - 8.56s
• MyISAM - 1.84s
• PGSQL - 8.4s

题

为什么InnoDB和MyISAM在完成全表扫描方面的时间相距甚远？我只是在MySQL配置中遗漏了一些明显的东西吗？我已经使用MySQL多年了,并且只要我的问题被限制在"索引可以解决这个"问题的集合中就没有问题.

分区显然也会解决这个问题,但代价要高得多.

作为参考,这是MySQL和PGSQL的配置文件:

MYSQL CONFIG

[client]
port=3306

[mysql]
default-character-set=utf8

[mysqld]
port=3306
basedir="C:/Program Files/MySQL/MySQL Server 5.0/"
datadir="C:/Program Files/MySQL/MySQL Server 5.0/Data/"
default-character-set=utf8
default-storage-engine=INNODB
log="c:/logs/mysql/mysqld.log"
sql-mode="STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"
max_connections=700
query_cache_size=0M
table_cache=1400
tmp_table_size=16M
thread_cache_size=34

myisam_max_sort_file_size=100G
myisam_sort_buffer_size=8M
key_buffer_size=200M
read_buffer_size=64K
read_rnd_buffer_size=256K
sort_buffer_size=208K

innodb_additional_mem_pool_size=2M
innodb_flush_log_at_trx_commit=1
innodb_log_buffer_size=1M
innodb_buffer_pool_size=200M
innodb_log_file_size=18M
innodb_thread_concurrency=10

PGSQL CONFIG

listen_addresses = '*'        # what IP address(es) to listen on;
                # comma-separated list of addresses;
                # defaults to 'localhost', '*' = all
                # (change requires restart)
port = 5432                # (change requires restart)
max_connections = 100            # (change requires restart)

shared_buffers = 32MB            # min 128kB
                # (change requires restart)
temp_buffers = 12MB            # min 800kB
maintenance_work_mem = 32MB        # min 1MB

log_destination = 'stderr'        # Valid values are combinations of
                # stderr, csvlog, syslog and eventlog,
                # depending on platform.  csvlog
                # requires logging_collector to be on.

logging_collector = on        # Enable capturing of stderr and csvlog
                # into log files. Required to be on for
                # csvlogs.
                # (change requires restart)

log_line_prefix = '%t'            # special values:
                #   %u = user name
                #   %d = database name
                #   %r = remote host and port
                #   %h = remote host
                #   %p = process ID
                #   %t = timestamp without milliseconds
                #   %m = timestamp with milliseconds
                #   %i = command tag
                #   %c = session ID
                #   %l = session line number
                #   %s = session start timestamp
                #   %v = virtual transaction ID
                #   %x = transaction ID (0 if none)
                #   %q = stop here in non-session
                #        processes
                #   %% = '%'
                # e.g. '<%u%%%d> '

datestyle = 'iso, mdy'
lc_messages = 'English_United States.1252'            # locale for system error message
                # strings
lc_monetary = 'English_United States.1252'            # locale for monetary formatting
lc_numeric = 'English_United States.1252'            # locale for number formatting
lc_time = 'English_United States.1252'                # locale for time formatting

default_text_search_config = 'pg_catalog.english'

另外要了解这些数据集在MySQL中的实际大小,这里有一个show table status\G on,如果有帮助的话:

*************************** 1. row ***************************
           Name: sample_innodb
         Engine: InnoDB
        Version: 10
     Row_format: Compact
           Rows: 5000205
 Avg_row_length: 100
    Data_length: 500154368
Max_data_length: 0
   Index_length: 149700608
      Data_free: 0
 Auto_increment: NULL
    Create_time: 2010-02-02 17:27:50
    Update_time: NULL
     Check_time: NULL
      Collation: utf8_general_ci
       Checksum: NULL
 Create_options:
        Comment: InnoDB free: 497664 kB

*************************** 2. row ***************************
           Name: sample_isam
         Engine: MyISAM
        Version: 10
     Row_format: Dynamic
           Rows: 5000000
 Avg_row_length: 72
    Data_length: 360006508
Max_data_length: 281474976710655
   Index_length: 1024
      Data_free: 0
 Auto_increment: NULL
    Create_time: 2010-02-02 17:27:50
    Update_time: 2010-02-02 17:37:23
     Check_time: NULL
      Collation: utf8_general_ci
       Checksum: NULL
 Create_options:
        Comment:

Answer 1

在我的服务器上使用您的配置我的基本性能如下:

• InnoDB:5.71s
• MyISAM:2.50秒

这在我的书中并不坏,但它可以使用一些调整.

您可以在以下几个方面改进InnoDB的性能:

增加 innodb_buffer_pool_size

• 这是唯一最重要的InnoDB配置变量.理想情况下,它应该是专用于MySQL 和 InnoDB 的服务器上70-80%的可用RAM .
• 增加innodb_buffer_pool_size我的服务器上,以2G(足够多的该测试)减少 InnoDB的时间4.60s

将id设为PRIMARY KEY

• InnoDB基于PRIMARY KEY对其数据进行聚类.当你没有声明一个时,InnoDB会隐式生成一个随机的.具有顺序主键(id)比随机主键快.
• 将id作为我服务器上的PRIMARY KEY 减少了InnoDB时间到3.80秒

升级你的MySQL/InnoDB

MySQL 5.1+ MySQL支持可插拔存储引擎.特别是新的InnoDB插件.

新的InnoDB引擎提供了许多性能增强,可能会对这种特定类型的查询产生重大影响.

值得注意的是

• 从MySQL 5.1.38开始,InnoDB插件包含在MySQL中
• 从MySQL 5.1.43开始,InnoDB插件不仅包含在内,而且是MySQL的默认引擎