删除数据的问题

Question

删除数据的问题

Dio*_*aim 6 mysql foreign-key delete

我一直试图删除我的 MySQL 数据库上的一些信息。

我从来没有用过这么大的数据库，当我开发这个应用程序时，我没想到它会变得这么大（对你们来说可能很小，但对我来说它很大）。

我的想法是：

我有一张带有“mos”托管对象（PK moid）的表（图 1）
我有一张带有“句号”的表格（PK periodid）（图 1）
我有一个将“Period”与 MO（PK moperiodid, FKs periodid, moid）连接起来的表（图 1）
我有 200 个使用moperiodidas FK 的表（表的数量根据用户输入而变化）（图 2）
然后，当我想清理“历史数据”时，我只需从 MO 或 PERIOD 中删除级联。

我就是这么想的，但在我的现实世界中，情况并不好。

对于 Selects 和 Inserts 一切正常，但是我第一次想开始清理数据时，主要想法是只保留最后 X 天（也根据用户输入进行配置）。

我每天有 3094555 个新的 moperiods，即使只删除一个也不起作用：

0   13  22:46:19    delete from parser_customer_period_mo where id = 1  Error Code: 1205. Lock wait timeout exceeded; try restarting transaction    51.169 sec

Run Code Online (Sandbox Code Playgroud)

每次尝试后，InnoDB 每秒写入/读取数会疯狂 1 小时。（图 3）

我不知道如何删除旧信息，这是一个小系统，我正在构建30个系统中的第二个，第二个将有12倍的数据量。

图 1：

图 2：

图 3：

Answer 1

Pau*_*ite 0

_{答案最初由作者添加到问题中}

我解决了今天的问题，但我不知道我的解决方法是否足以满足未来的需要。

我的解决方法是：

删除所有 FK
我没有删除旧数据（~20GB），而是选择“进入新的 MyISAM 表（~40GB）”，这对我来说足够快了。
删除旧表并将新表重命名为旧名称。

显然，我使用 MyISAM 获得了更好的 JOIN 性能（用户使用该界面创建的所有查询将如下所示）：

SELECT 
         SUBSTRING_INDEX(SUBSTRING_INDEX(m.moname,'LabelName=',-1),',',1) as 'MO Name' ,p.datetime_start,(   ( zzce_SOMECOUNTER.value  +  zzce_SOMEOTHERCOUNTER.value  -  zzce_SOMECOUNTERUra.value  -  zzce_SOMEOTHERCOUNTERUra.value  +  zzce_ANOTHERCOUNTER.value )  ) as value
     FROM
         parser_customer_mos AS m
             INNER JOIN
         parser_customer_period_mo pm ON m.id = pm.moid    
         INNER JOIN
         parser_period p ON pm.periodid = p.id    
          INNER JOIN zzce_SOMECOUNTER ON zzce_SOMECOUNTER.moperiod = pm.id  INNER JOIN zzce_SOMEOTHERCOUNTER ON zzce_SOMEOTHERCOUNTER.moperiod = pm.id  INNER JOIN zzce_SOMECOUNTERUra ON zzce_SOMECOUNTERUra.moperiod = pm.id  INNER JOIN zzce_SOMEOTHERCOUNTERUra ON zzce_SOMEOTHERCOUNTERUra.moperiod = pm.id  INNER JOIN zzce_ANOTHERCOUNTER ON zzce_ANOTHERCOUNTER.moperiod = pm.id  
     WHERE
         m.id IN (SELECT 
                 id
             FROM
                 parser_customer_mos
             WHERE
                 moname LIKE 'ManagedElement=1,RncFunction=1,UtranCell=MGCBHE165OU_%' or moname LIKE 'ManagedElement=1,RncFunction=1,UtranCell=PRCCTA010OU_%' or moname LIKE 'ManagedElement=1,RncFunction=1,UtranCell=PRCCTA055OU_%')
             AND 
      pm.periodid IN (SELECT 
                 id
             FROM
                 parser_period
             WHERE
                 datetime_start BETWEEN '2015-12-12 00:00:00' AND '2015-12-12 17:44:28') ORDER BY datetime_start asc;

Run Code Online (Sandbox Code Playgroud)

想法是：我每天多次（PERIOD）为许多对象（MO）设置许多计数器组（“计数器表”）。有些 MO 可以使所有计数器处于活动状态，有些则只能使少数计数器处于活动状态。

CREATE TABLE IF NOT EXISTS `perf_customer_rnc`.`parser_customer_mos` (
      `id` BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT COMMENT '',
      `moname` VARCHAR(200) NOT NULL DEFAULT 'asd' COMMENT '',
      `nename` VARCHAR(10) NOT NULL DEFAULT 'asd' COMMENT '',
      PRIMARY KEY (`moname`, `nename`)  COMMENT '',
      UNIQUE INDEX `id_UNIQUE` (`id` ASC)  COMMENT '')
    ENGINE = InnoDB
    AUTO_INCREMENT = 27358
    DEFAULT CHARACTER SET = latin1

CREATE TABLE IF NOT EXISTS `perf_customer_rnc`.`parser_period` (
  `id` BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT COMMENT '',
  `datetime_start` DATETIME NOT NULL DEFAULT '2001-01-01 00:01:00' COMMENT '',
  `datetime_end` DATETIME NOT NULL DEFAULT '2001-01-01 00:01:00' COMMENT '',
  `datetime_gmt_start_minutes` SMALLINT(6) NOT NULL DEFAULT '0' COMMENT '',
  `datetime_gmt_end_minutes` SMALLINT(6) NOT NULL DEFAULT '0' COMMENT '',
  `measurement_elapsed_minutes` SMALLINT(6) NULL DEFAULT NULL COMMENT '',
  PRIMARY KEY (`datetime_start`, `datetime_end`, `datetime_gmt_start_minutes`, `datetime_gmt_end_minutes`)  COMMENT '',
  UNIQUE INDEX `id_UNIQUE` (`id` ASC)  COMMENT '')
ENGINE = InnoDB
AUTO_INCREMENT = 9904
DEFAULT CHARACTER SET = latin1

CREATE TABLE `perf_customer_rnc`.`parser_customer_period_mo` (
  `id` BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT COMMENT '',
  `moid` BIGINT(20) UNSIGNED NOT NULL COMMENT '',
  `periodid` BIGINT(20) UNSIGNED NOT NULL COMMENT '',
  PRIMARY KEY (`moid`, `periodid`)  COMMENT '',
  UNIQUE INDEX `id_UNIQUE` (`id` ASC)  COMMENT '',
  INDEX `moid_idx` (`moid` ASC)  COMMENT '',
  INDEX `periodid_idx` (`periodid` ASC)  COMMENT '',
  CONSTRAINT `mofk`
    FOREIGN KEY (`moid`)
    REFERENCES `perf_customer_rnc`.`parser_customer_mos` (`id`)
    ON DELETE CASCADE
    ON UPDATE NO ACTION,
  CONSTRAINT `periodfk`
    FOREIGN KEY (`periodid`)
    REFERENCES `perf_customer_rnc`.`parser_period` (`id`)
    ON DELETE CASCADE
    ON UPDATE NO ACTION)
ENGINE = InnoDB
AUTO_INCREMENT = 42331658
DEFAULT CHARACTER SET = latin1

CREATE TABLE IF NOT EXISTS `perf_customer_rnc`.`zzce_CounterName` (
  `moperiod` BIGINT(20) UNSIGNED NOT NULL COMMENT '',
  `value` INT(11) NOT NULL COMMENT '',
  PRIMARY KEY (`moperiod`)  COMMENT '',
  CONSTRAINT `fk_CounterName`
    FOREIGN KEY (`moperiod`)
    REFERENCES `perf_customer_rnc`.`parser_customer_period_mo` (`id`)
    ON DELETE CASCADE
    ON UPDATE NO ACTION)
ENGINE = InnoDB
DEFAULT CHARACTER SET = latin1

CREATE TABLE IF NOT EXISTS `perf_customer_rnc`.`zzce_someOtherCounterName` (
  `moperiod` BIGINT(20) UNSIGNED NOT NULL COMMENT '',
  `value` VARCHAR(112) NOT NULL COMMENT '',
  PRIMARY KEY (`moperiod`)  COMMENT '',
  CONSTRAINT `fk_someOtherCountername`
    FOREIGN KEY (`moperiod`)
    REFERENCES `perf_customer_rnc`.`parser_customer_period_mo` (`id`)
    ON DELETE CASCADE
    ON UPDATE NO ACTION)
ENGINE = InnoDB
DEFAULT CHARACTER SET = latin1

CREATE TABLE IF NOT EXISTS `perf_customer_rnc`.`zzce_someOtherCounterName2` (
  `moperiod` BIGINT(20) UNSIGNED NOT NULL COMMENT '',
  `value` BIT(1) NOT NULL COMMENT '',
  `value2` BIT(1) NOT NULL COMMENT '',
  `value3` BIT(1) NOT NULL COMMENT '',
  `value4` BIT(1) NOT NULL COMMENT '',
  `value5` BIT(1) NOT NULL COMMENT '',
  `value6` BIT(1) NOT NULL COMMENT '',
  `value7` BIT(1) NOT NULL COMMENT '',
  `value8` BIT(1) NOT NULL COMMENT '',
  `value9` BIT(1) NOT NULL COMMENT '',
  `value10` BIT(1) NOT NULL COMMENT '',
  `value11` BIT(1) NOT NULL COMMENT '',
  `value12` BIT(1) NOT NULL COMMENT '',
  `value13` BIT(1) NOT NULL COMMENT '',
  `value14` BIT(1) NOT NULL COMMENT '',
  `value15` BIT(1) NOT NULL COMMENT '',
  `value16` BIT(1) NOT NULL COMMENT '',
  `value17` BIT(1) NOT NULL COMMENT '',
  `value18` BIT(1) NOT NULL COMMENT '',
  `value19` BIT(1) NOT NULL COMMENT '',
  `value20` BIT(1) NOT NULL COMMENT '',
  `value21` BIT(1) NOT NULL COMMENT '',
  `value22` BIT(1) NOT NULL COMMENT '',
  PRIMARY KEY (`moperiod`)  COMMENT '',
  CONSTRAINT `fk_SomeCounterName`
    FOREIGN KEY (`moperiod`)
    REFERENCES `perf_customer_rnc`.`parser_customer_period_mo` (`id`)
    ON DELETE CASCADE
    ON UPDATE NO ACTION)
ENGINE = InnoDB
DEFAULT CHARACTER SET = latin1

Run Code Online (Sandbox Code Playgroud)

第一次审判的主持人是：

主机：Linode 8GB - 8 GB RAM - 192 GB SSD - 6 个 vCPU

较大的会使用一些其他的 VPS。

my.cnf 我没有做太多事情。

#
# The MySQL database server configuration file.
#
# You can copy this to one of:
# - "/etc/mysql/my.cnf" to set global options,
# - "~/.my.cnf" to set user-specific options.
#
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# For explanations see
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html

# This will be passed to all mysql clients
# It has been reported that passwords should be enclosed with ticks/quotes
# escpecially if they contain "#" chars...
# Remember to edit /etc/mysql/debian.cnf when changing the socket location.
[client]
port            = 7979
socket          = /var/run/mysqld/mysqld.sock

# Here is entries for some specific programs
# The following values assume you have at least 32M ram

# This was formally known as [safe_mysqld]. Both versions are currently parsed.
[mysqld_safe]
socket          = /var/run/mysqld/mysqld.sock
nice            = 0

[mysqld]
#
# * Basic Settings
#
user            = mysql
pid-file        = /var/run/mysqld/mysqld.pid
socket          = /var/run/mysqld/mysqld.sock
port            = 7979
basedir         = /usr
datadir         = /var/lib/mysql
tmpdir          = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking
#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
#bind-address           = 127.0.0.1
#
# * Fine Tuning
#
key_buffer              = 16M
max_allowed_packet      = 16M
thread_stack            = 192K
thread_cache_size       = 8
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
myisam-recover         = BACKUP
#max_connections        = 100
#table_cache            = 64
#thread_concurrency     = 10
#
# * Query Cache Configuration
#
query_cache_limit       = 1M
query_cache_size        = 16M
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
#general_log_file        = /var/log/mysql/mysql.log
#general_log             = 1
#
# Error log - should be very few entries.
#
log_error = /var/log/mysql/error.log
#
# Here you can see queries with especially long duration
#log_slow_queries       = /var/log/mysql/mysql-slow.log
#long_query_time = 2
#log-queries-not-using-indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
#       other settings you may need to change.
#server-id              = 1
#log_bin                        = /var/log/mysql/mysql-bin.log
expire_logs_days        = 10
max_binlog_size         = 100M
#binlog_do_db           = include_database_name
#binlog_ignore_db       = include_database_name
#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
#
# * Security Features
#
# Read the manual, too, if you want chroot!
# chroot = /var/lib/mysql/
#
# For generating SSL certificates I recommend the OpenSSL GUI "tinyca".
#
# ssl-ca=/etc/mysql/cacert.pem
# ssl-cert=/etc/mysql/server-cert.pem
# ssl-key=/etc/mysql/server-key.pem



[mysqldump]
quick
quote-names
max_allowed_packet      = 16M

[mysql]
#no-auto-rehash # faster start of mysql but no tab completition

[isamchk]
key_buffer              = 16M

#
# * IMPORTANT: Additional settings that can override those from this file!
#   The files must end with '.cnf', otherwise they'll be ignored.
#
!includedir /etc/mysql/conf.d/

Run Code Online (Sandbox Code Playgroud)

归档时间：	9 年，9 月前
查看次数：	630 次
最近记录：	8 年，6 月前