使用python将数据从csv复制到postgresql

Question

我在Windows 7 64位.我有一个csv文件'data.csv'.我想通过python脚本将数据导入postgresql表'temp_unicommerce_status'.

我的剧本是:

import psycopg2
conn = psycopg2.connect("host='localhost' port='5432' dbname='Ekodev' user='bn_openerp' password='fa05844d'")
cur = conn.cursor()
cur.execute("""truncate table "meta".temp_unicommerce_status;""")
cur.execute("""Copy temp_unicommerce_status from 'C:\Users\n\Desktop\data.csv';""")
conn.commit()
conn.close()

我收到了这个错误

Traceback (most recent call last):
  File "C:\Users\n\Documents\NetBeansProjects\Unicommerce_Status_Update\src\unicommerce_status_update.py", line 5, in <module>
cur.execute("""Copy temp_unicommerce_status from     'C:\\Users\\n\\Desktop\\data.csv';""")
psycopg2.ProgrammingError: must be superuser to COPY to or from a file
HINT:  Anyone can COPY to stdout or from stdin. psql's \copy command also works for anyone.

Answer 1

使用copy_from游标方法

f = open(r'C:\Users\n\Desktop\data.csv', 'r')
cur.copy_from(f, temp_unicommerce_status, sep=',')
f.close()

该文件必须作为对象传递.

由于您要从csv文件进行处理,因此必须指定分隔符,因为默认值是制表符

Answer 2

#sample of code that worked for me

import psycopg2 #import the postgres library

#connect to the database
conn = psycopg2.connect(host='localhost',
                       dbname='database1',
                       user='postgres',
                       password='****',
                       port='****')  
#create a cursor object 
#cursor object is used to interact with the database
cur = conn.cursor()

#create table with same headers as csv file
cur.execute("CREATE TABLE IF NOT EXISTS test(**** text, **** float, **** float, **** 
text)")

#open the csv file using python standard file I/O
#copy file into the table just created 
with open('******.csv', 'r') as f:
next(f) # Skip the header row.
    #f , <database name>, Comma-Seperated
    cur.copy_from(f, '****', sep=',')
    #Commit Changes
    conn.commit()
    #Close connection
    conn.close()


f.close()

Answer 3

我解决此问题的方法特别是使用psychopg2游标类函数copy_expert（文档：http ://initd.org/psycopg/docs/cursor.html ）。copy_expert允许您使用STDIN，因此无需为postgres用户发出超级用户特权。然后，您对文件的访问取决于客户端（linux / windows / mac）用户对文件的访问

从Postgres COPY Docs（https://www.postgresql.org/docs/current/static/sql-copy.html）：

请勿将COPY与psql指令\ copy混淆。\ copy调用COPY FROM STDIN或COPY TO STDOUT，然后将数据提取/存储在psql客户端可访问的文件中。因此，使用\ copy时，文件的可访问性和访问权限取决于客户端而不是服务器。

您还可以严格设置访问权限，以访问development_user主文件夹和App文件夹。

csv_file_name = '/home/user/some_file.csv'
sql = "COPY table_name FROM STDIN DELIMITER '|' CSV HEADER"
cursor.copy_expert(sql, open(csv_file_name, "r"))