处理上传的图像zip时的HTTP 504

Question

我是网络开发的新手,我正在使用Django开发一个基本的图库应用程序(学习练习).我设置好所以我可以一次上传一个完整的图像来创建一个新专辑.这一切似乎都运行正常,但是当上传的文件特别大时,我收到HTTP 504错误.

我收集(如果我错了请纠正我)这个错误意味着我的应用程序太慢而无法返回HTTP响应.我猜这是因为解压缩和处理(在数据库中创建Pic对象并创建缩略图)所有图像需要很长时间.

有没有办法在后台执行处理时返回响应(比如某些中间页面) - 也许使用线程？处理这个问题的正确方法是什么？是时候开始学习Javascript/AJAX吗？

谢谢!

---

楷模:

from django.db import models
from blog.models import Post

class Album(models.Model):
    title = models.CharField(max_length=128)
    slug = models.SlugField()
    description = models.TextField()
    parent = models.ForeignKey('self', null=True, blank=True)

    pub = models.BooleanField()
    date_created = models.DateTimeField(auto_now_add=True)
    date_published = models.DateTimeField(null=True, blank=True)
    date_modified = models.DateTimeField(auto_now=True)

    def __unicode__(self):
        return self.title

class Pic(models.Model):
    image = models.ImageField(upload_to='pics/%Y/%m') 
    title = models.CharField(max_length=128)
    caption = models.TextField(blank=True, null=True)
    albums = models.ManyToManyField('Album', null=True, blank=True)
    posts = models.ManyToManyField(Post, blank=True, null=True)

    date_taken = models.DateTimeField(null=True, blank=True) 
    date_uploaded = models.DateTimeField(auto_now_add=True) 
    date_modified = models.DateTimeField(auto_now=True)

    def __unicode__(self):
        return self.title

视图:

_{我正在手动执行此操作,因为当我开始时我没有找到Django管理员.我认为在这里使用管理员自定义可能更好.}

def new_album(request):
    if request.method == "POST":
        form = AlbumForm(request.POST, request.FILES)
        if form.is_valid():
            from gallery.pic_handlers import handle_uploaded_album
            pics = handle_uploaded_album(request.FILES['pic_archive'])
            a = form.save()
            a.slug = slugify(a.title)
            a.save()
            for pic in pics:
                pic.albums.add(a)
            return HttpResponseRedirect('/gallery/album/%s/' % a.slug)
    else:
        form = AlbumForm()

    return render_to_response('new_album.html', {
        'form' : form,
    }, context_instance = RequestContext(request))

附加处理:

def handle_uploaded_album(pic_archive):
    destination = open(join(settings.MEDIA_ROOT,pic_archive.name), 'wb+')
    for chunk in pic_archive.chunks():
        destination.write(chunk)
    destination.close()

    today = datetime.date.today()
    save_path = 'pics/{0}/{1:02}/'.format(today.year, today.month)
    tmp_path = 'tmp/'
    z = zipfile.ZipFile(join(settings.MEDIA_ROOT,pic_archive.name), 'r')
    pics = []
    for member in z.namelist():
        if '/' in member or '\\' in member: 
            # don't deal with any directories inside the zip
            # this also solves the '__MACOSX' issue
            continue
        if splitext(member)[1] in IMG_EXT:
            z.extract(member,join(settings.MEDIA_ROOT,tmp_path))
            im = File(open(join(settings.MEDIA_ROOT,tmp_path,member), 'rb'))
            # create a Pic from this file
            pic = Pic()
            pic.title = member
            pic.image.save(
                join(save_path, member),
                im,
                True)
            create_thumbnails(pic)
            im.close()
            # remove extracted images
            remove(join(settings.MEDIA_ROOT,tmp_path,member))

            # TODO: save date taken if available
            pics.append(pic)

    z.close()
    remove(join(settings.MEDIA_ROOT,pic_archive.name))

    return pics

def create_thumbnails(pic):
    fname, ext = splitext(pic.image.path)
    img = Image.open(pic.image.path)

    img.thumbnail((512,512), Image.ANTIALIAS)
    img.save(fname + '_m' + ext)

    img.thumbnail((128,128), Image.ANTIALIAS)
    img.save(fname + '_s' + ext)

Answer 1

诸如此处理之类的长任务需要花费太多时间,并且您的客户端和/或代理将超时 - 这是您看到的504错误.

你不应该以这种方式长期工作!

正如您在最后正确提出的那样,您需要一种方法来分离长时间的执行 - 通过异步队列系统,例如芹菜.这样,您可以立即将答案返回给客户,而后端则异步运行作业.

你应该看看下面的一个:

• Django长期运行线程/处理的异步任务
• 使用Django长时间运行任务
• 在Django长期投票

由于django-celery绝对是最好的选择,下一步就是了解它; 围绕它也有很多SO问题

如果您想确定它的处理失败,而不是上传,只需尝试禁用所有处理并直接返回到您的客户端.如果它仍然失败,您还需要调整您的网络服务器,以便它不会超时!