Python将一组.csv文件加载到MySQL中

Question

我使用以下命令将多个.csv文件加载到Mysql数据库中,但我没有收到任何错误(IDLE窗口)并且数据未加载

这是错误的脚本

#!C:\Python27\python.exe

import MySQLdb
import os
import string

# Open database connection
db = MySQLdb.connect (host="localhost",port=3307,user="root",\
                      passwd="gamma123",db="test")

cursor=db.cursor()


l = os.listdir(".")
for file_name in l:
        print file_name
        cursor=db.cursor()
        if (file_name.find("DIV.csv")>-1):
           #Query under testing
            sql = """LOAD DATA LOCAL INFILE file_name \
            INTO TABLE system_work \
            FIELDS TERMINATED BY ',' \
            OPTIONALLY ENCLOSED BY '"'  \
            LINES TERMINATED BY '\r\n' \
            IGNORE 1 LINES;;"""
try:
            # Execute the SQL command
            cursor.execute(sql)
            # Commit your changes in the database
            db.commit()
except:
            # Rollback in case there is any error
            db.rollback()

# disconnect from server
db.close()

但是当我尝试使用以下python脚本加载单个文件时,它的工作正常.请帮忙....

#!C:\Python27\python.exe

import MySQLdb
import os
import string

# Open database connection
db = MySQLdb.connect (host="localhost",port=3307,user="root",\
                      passwd="gamma123",db="test")

cursor=db.cursor()

#Query under testing
sql = """LOAD DATA LOCAL INFILE 'Axle.csv' \
      INTO TABLE system_work \
      FIELDS TERMINATED BY ',' \
      OPTIONALLY ENCLOSED BY '"'  \
      LINES TERMINATED BY '\r\n' \
      IGNORE 1 LINES;;"""

try:
   # Execute the SQL command
   cursor.execute(sql)
   # Commit your changes in the database
   db.commit()
except:
   # Rollback in case there is any error
   db.rollback()

# disconnect from server
db.close()

Answer 1

您需要将文件名插入到SQL字符串中; 您只是将文字文​​本发送file_name到服务器.您可以使用该str.format()方法,{}然后可以使用您选择的变量替换任何占位符.

您还必须缩进try和except块在for循环内:

sql = """LOAD DATA LOCAL INFILE '{}'
INTO TABLE system_work
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\\r\\n'
IGNORE 1 LINES;;"""

for file_name in l:
    print file_name
    if file_name.endswith('DIV.csv'):
        try:
            cursor = db.cursor()
            cursor.execute(sql.format(file_name))
            db.commit()
        except Exception:
            # Rollback in case there is any error
            db.rollback()

该cursor.execute()方法通过插值变量传递给sql字符串file_name.在{}第一行(部分LOAD DATA LOCAL INFILE '{}')将通过价值来代替file_name传递SQL语句到MySQL之前.

我还简化了文件名测试; 大概如果文件​​名以.结尾就足够了DIV.csv.

请注意,使用该mysqlimport实用程序可能更容易; 你可以用以下方法获得完全相同的结果:

mysqlimport --fields-terminated-by=, --fields-optionally-enclosed-by=\" \
            --local --lines-terminated-by=\r\n --user=root --password=gamma123 \
            test *DIV.csv