如何导入数据库,更新已更改的产品,删除已删除的产品

Question

我对基本的MySQL"好",但这是"我的头脑"!

目标:

• 导入数据库
• 更新已更改的产品
• 删除已删除的产品
• 快速有效

数据库表是巨大的,速度是一个问题.

没有MyISAM是inoDB会更快吗？每个数据库都在一个唯一的表中.

我被赋予了这个作为我正在尝试做的事情的起点:

CREATE TABLE `table` LIKE LiveTable
LOAD DATA INFILE..... INTO `table`
UPDATE `table`  SET delete=1; -- Set the delete field to true  because it will not have been updated
UPDATE `table` INNER JOIN`table`ON `LiveTable.ID`=`table.ID`
SET LiveTable.Col1=table.Col1, LiveTable.Col2=table.Col2….. delete=0
INSERT INTO LiveTable(ID,Col1,Col2,…  delete=0)
SELECT ID,Col1,Col2,...FROM `table`
LEFT JOIN LiveTable 
ON table.ID = LiveTable.ID
WHERE LiveTable.ID IS NULL
DELETE FROM LiveTableWHERE delete = 0
EMPTY TABLE `table`

---

> CREATE TABLE `product_table`   (
>      `programname` VARCHAR(100) NOT NULL,
>      `name`        VARCHAR(160) NOT NULL,
>      `keywords`    VARCHAR(300) NOT NULL,
>      `description` TEXT NOT NULL,
>      `sku`         VARCHAR(100) NOT NULL,
>      -- This is the only "unique identifier given, none will be duplicates"
>      `price`       DECIMAL(10, 2) NOT NULL,
>      `created`     TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
>      `updatedat`   TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00',
>      `delete`      TINYINT(4) NOT NULL DEFAULT '0',
>      PRIMARY KEY (`sku`)   ) ENGINE=myisam DEFAULT CHARSET=latin1;
> 
> CREATE TABLE IF NOT EXISTS `temptable` LIKE `product_table`;
> 
> TRUNCATE TABLE `temptable`; -- Remove data from temp table if for some
> reason it has data in it.
> 
> LOAD DATA LOW_PRIORITY LOCAL INFILE "catalog.csv" INTO TABLE
> `temptable`  FIELDS TERMINATED BY "," OPTIONALLY ENCLOSED BY """"
> LINES TERMINATED BY "\n"  IGNORE 1 LINES (`PROGRAMNAME`, `NAME`,
> `KEYWORDS`, `DESCRIPTION`, `SKU`, `PRICE`);
> 
> 
> UPDATE `temptable` SET    `delete` = 1; -- Set the delete field to
> true UPDATE `temptable` ttable
>        INNER JOIN `product_table` mtable
>          ON ( mtable.sku = ttable.sku ) SET    mtable.programname = ttable.programname,
>        mtable.name = ttable.name,
>        mtable.keywords = ttable.keywords,
>        mtable.description = ttable.description,
>        mtable.sku = ttable.sku,
>        mtable.price = ttable.price,
>        mtable.created = ttable.created,
>        mtable.updatedat = NOW(),-- Set Last Update
>        mtable.delete = 0; -- Set Delete to NO
> 
> -- Not sure what this is for...  I'm LOST at this part...   
> INSERT INTO `product_table` VALUES      (`programname`,
>              `name`,
>              `keywords`,
>              `description`,
>              `sku`,
>              `price`,
>              `created`,
>              `updatedat`,
>              `delete`);
> 
> -- This type of join requires alias as far as I know? 
> SELECT `programname`,
>        `name`,
>        `keywords`,
>        `description`,
>        `sku`,
>        `price`,
>        `created`,
>        `updatedat`,
>        `delete` FROM   `temptable` tmptable
>        LEFT JOIN `product_table` maintbl
>          ON tmptable.sku = maintbl.sku WHERE  maintbl.sku IS NULL;
> 
> DELETE FROM `product_table` WHERE  `delete` = 0;
> 
> TRUNCATE TABLE `temptable`; `` remove all the data from temporary
> table.

Answer 1

我自己在这里回答了这个问题： https://dba.stackexchange.com/questions/16197/innodb-update-slow-need-a-better-option/16283#16283

利用我从这里、网络和几个互联网聊天室收到的信息，我想出了一些建议。网络来源：http://www.softwareprojects.com/resources/programming/t-how-to-use-mysql-fast-load-data-for-updates-1753.html

[DEMO][1] http://sqlfiddle.com/#!2/4ebe0/1

其过程是：

Import into a new temp table.
Update The old table information with information in Temp table.
Insert new data into the table. (Real world I'm making a new CSV file and using LOAD INTO for the insert)
delete everything that is no longer in the data feed.
delete the temp table.

这似乎是迄今为止最快的过程。

让我知道你的意见是什么。