如何将.csv文件中的数据传输到django中的sqlite数据库？

Question

这是我的models.py

from django.db import models

class School(models.Model):    
    school = models.CharField(max_length=300)
    def __unicode__(self):
        return self.school
class Lawyer(models.Model):
    firm_url = models.URLField('Bio', max_length=200)
    firm_name = models.CharField('Firm', max_length=100)
    first = models.CharField('First Name', max_length=50)
    last = models.CharField('Last Name', max_length=50)
    year_graduated = models.IntegerField('Year graduated')
    school = models.CharField(max_length=300)
    school = models.ForeignKey(School)
    class Meta:
        ordering = ('?',)
   def __unicode__(self):
        return self.first    

来自csv文件的2个示例行:

"http://www.graychase.com/aabbas,Gray & Chase LLP, Amr A ,Abbas,The George Washington University Law School, 2005"
"http://www.graychase.com/kadam,Gray & Chase LLP, Karin ,Adam,Ernst Moritz Arndt University Greifswald, 2004"

谢谢.

编辑

你能提供一些关于这个脚本的更详细的信息吗？一些问题:

我的应用程序在

C:.../Documents/PROJECTS/Django/sw2/wkw2.

这条路已经在PYTHONPATH我还需要这条线吗？如果是这样,我会这样输入吗？

sys.path.append('C:\\sw2')

怎么os.environ办？我试着阅读文档,但我不明白.

os.environ['DJANGO_SETTINGS_MODULE'] = 'sw2.settings'

from django.core.management import setup_environ
from sw2 import settings
from sw2.wkw2.models import *

import csv

dataReader = csv.reader(open('csvtest1.csv'), delimiter=',', quotechar='"')

for row in dataReader:
    lawyer=Lawyer()
    lawyer.firm_url=row[0]
    lawyer.firm_name=row[1]
    lawyer.first=row[2]
    lawyer.last=row[3]
    lawyer.school=row[4]
    lawyer.year_graduated=row[5]
    lawyer.save()

谢谢!

根据celopes的回答编辑:

celopes:

我看到你的回答有点晚了.我一直在尝试用shell更新数据库中的数据库

>>> p1 = Lawyer(school = "The George Washington University Law School", last = "Babbas", firm_url = "http://www.graychase.com/babbas", year_graduated = 2005, firm_name=  "Gray & Chase", first= "Amr A")

但我一直得到整数错误.最后,我意识到学校需要成为school_id,即形式,

>>> p1 = Lawyer(school_id = 1, last = "Babbas", firm_url = "http://www.graychase.com/babbas", year_graduated = 2005, firm_name=  "Gray & Chase", first= "Amr A")

由此我意识到,我需要知道每所学校的学校ID以更新律师表.由于那是不可能的,我决定删除它,ForeignKey因为我不知道如何解决这个问题.(我是Python和Django的新手.)

今天早上我看到了你的回答.现在我在开发服务器中更改了我的模型,我只有一个表:律师.我想这就是我将要使用的.我再次道歉,因为没有早点看到你的答案.

编辑12/14/09:

celopes:

再次感谢您的脚本.它解决了我的问题.在保存到数据库之前,我不需要将csv转换为json或其他格式.我做了一些改变.首先如前所述,我将模型改为律师.此外,您使用"名称"修复了重复的学校.但我有list_display,list_filter并且search_fields在admin.py和不断变化的领域名字引起了太多的错误.

class LawyerAdmin(admin.ModelAdmin):
    fieldsets = [
        ('Name',   {'fields': ['last', 'first', 'firm_name', 'firm_url', 'school', 'year_graduated']}),
        #('School', {'fields': ['school', 'year_graduated']}),
]
    list_display = ('last', 'first', 'firm_name', 'firm_url', 'school', 'year_graduated')
    list_filter = ['school', 'year_graduated']
    search_fields = ['last', 'school', 'firm_name']
    #search_fields = ['school__school']
    #search_fields = ['school__lawyer__last']

随着新models.py的简化脚本运作良好.这就是我正在使用的:

csv_filepathname="C:/Users/A/Documents/Projects/Django/sw2/wkw2/fixtures/data1.csv"
your_djangoproject_home="C:/Users/A/Documents/PROJECTS/Django/"

import sys,os
sys.path.append(your_djangoproject_home)
os.environ['DJANGO_SETTINGS_MODULE'] ='sw2.settings'

from sw2.wkw2.models import Lawyer

import csv
dataReader = csv.reader(open(csv_filepathname), delimiter=',', quotechar='"')

for row in dataReader:
    lawyer=Lawyer()
    lawyer.firm_url=row[0]
    lawyer.firm_name=row[1]
    lawyer.first=row[2]
    lawyer.last=row[3]
    lawyer.school=row[4]
    lawyer.year_graduated=row[5]
    lawyer.save()

我还删除了每行的引号.我注意到,如果我把年份放在引号中,我得到整数错误,没有引号工作正常.你是如何使用引号的？

再次感谢,这非常有帮助.现在我必须让它在生产服务器中工作.

Answer 1

我使用这些数据作为测试创建了一个完整的脚本:

"http://www.graychase.com/aabbas","Gray & Chase LLP","Amr A","Abbas","The George Washington University Law School","2005"
"http://www.graychase.com/kadam","Gray & Chase LLP","Karin","Adam","Ernst Moritz Arndt University Greifswald","2004"

请注意,您在上面举例说明的CSV文件是错误的.csv文件阅读器将整行读取为条目,因为整行都在引号中.从csv文件中的每一行中删除开头和尾随引号,或者 - 像我一样 - 将每个不同的值括在引号中的行中.

以下是适用于以下脚本的模型:

from django.db import models

class School(models.Model):    
    name = models.CharField(max_length=300, unique=True)

    def __unicode__(self):
        return self.name

class Lawyer(models.Model):
    firm_url = models.URLField('Bio', max_length=200, unique=True)
    firm_name = models.CharField('Firm', max_length=100)
    first = models.CharField('First Name', max_length=50)
    last = models.CharField('Last Name', max_length=50)
    year_graduated = models.IntegerField('Year graduated')
    school = models.ForeignKey(School)

    def __unicode__(self):
        return self.first

这是将读取您的CSV文件的脚本(除非我得到您的项目名称sw2和应用程序wkw2错误,然后修复这些引用):

############ All you need to modify is below ############
# Full path and name to your csv file
csv_filepathname="C:/Users/A/Documents/Projects/Django/sw2/wkw2/fixtures/data.csv"
# Full path to the directory immediately above your django project directory
your_djangoproject_home="C:.../Documents/PROJECTS/Django/"
############ All you need to modify is above ############

import sys,os
sys.path.append(your_djangoproject_home)
os.environ['DJANGO_SETTINGS_MODULE'] ='sw2.settings'

from sw2.wkw2.models import School, Lawyer

import csv
dataReader = csv.reader(open(csv_filepathname), delimiter=',', quotechar='"')

old_school = None
for row in dataReader:
    if old_school != row[4]:
        old_school = row[4]
        school = School()
        school.name = old_school
        school.save()

dataReader = csv.reader(open(csv_filepathname), delimiter=',', quotechar='"')

for row in dataReader:
    lawyer=Lawyer()
    lawyer.firm_url=row[0]
    lawyer.firm_name=row[1]
    lawyer.first=row[2]
    lawyer.last=row[3]

    lawyer_school=School.objects.get(name=row[4])
    lawyer.school=lawyer_school

    lawyer.year_graduated=row[5]
    lawyer.save()

该脚本首先从CSV文件中的可用学校创建每个可能的学校.然后它再次运行CSV并创建每个律师.

我用测试数据运行了这个脚本.它工作正常并加载所有CSV数据.