标签: django-storage

我正在配置一个Django项目,该项目使用服务器文件系统来存储应用程序静态文件(STATIC_ROOT)和用户上传的文件(MEDIA_ROOT).

我现在需要在亚马逊的S3上托管所有内容,所以我为此创建了一个存储桶.使用django-storages与boto存储后端,我设法收集静态上传到S3斗:

MEDIA_ROOT = '/media/'
STATIC_ROOT = '/static/'

DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
AWS_ACCESS_KEY_ID = 'KEY_ID...'
AWS_SECRET_ACCESS_KEY = 'ACCESS_KEY...'
AWS_STORAGE_BUCKET_NAME = 'bucket-name'
STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'

然后,我遇到了一个问题:MEDIA_ROOT并且STATIC_ROOT没有在存储桶中使用,因此存储桶根目录包含静态文件和用户上载的路径.

那么我可以设置:

S3_URL = 'http://s3.amazonaws.com/%s' % AWS_STORAGE_BUCKET_NAME
STATIC_URL = S3_URL + STATIC_ROOT
MEDIA_URL = 'S3_URL + MEDIA_ROOT

并在模板中使用这些设置,但在S3中存储时,静态/媒体文件没有区别django-storages.

怎么做到这一点？

谢谢!

我正试图让django将静态文件上传到S3,但是我得到了403禁用错误,我不知道为什么.

完整Stacktrace:

Traceback (most recent call last):
  File "manage.py", line 14, in <module>
    execute_manager(settings)
  File "/home/levi/Projects/DoneBox/.virtualenv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 438, in execute_manager
    utility.execute()
  File "/home/levi/Projects/DoneBox/.virtualenv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 379, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/levi/Projects/DoneBox/.virtualenv/local/lib/python2.7/site-packages/django/core/management/base.py", line 191, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/levi/Projects/DoneBox/.virtualenv/local/lib/python2.7/site-packages/django/core/management/base.py", line 220, in execute
    output = self.handle(*args, **options)
  File "/home/levi/Projects/DoneBox/.virtualenv/local/lib/python2.7/site-packages/django/core/management/base.py", line 351, in handle
    return self.handle_noargs(**options)
  File "/home/levi/Projects/DoneBox/.virtualenv/local/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 89, in handle_noargs
    self.copy_file(path, prefixed_path, storage, **options)
  File "/home/levi/Projects/DoneBox/.virtualenv/local/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 184, in copy_file
    if not self.delete_file(path, prefixed_path, source_storage, **options):
  File "/home/levi/Projects/DoneBox/.virtualenv/local/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 115, …

我正在尝试切换到使用amazon s3来托管我们的django项目的静态文件.我正在使用django,boto,django-storage和django-compressor.当我在我的开发服务器上运行collect static时,我收到了错误

socket.error: [Errno 104] Connection reset by peer 

我所有静态文件的大小都是74MB,看起来不算太大.有没有人见过这个,或者有任何调试技巧？

这是完整的痕迹.

Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 443, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 382, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 196, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 232, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 371, in handle
    return self.handle_noargs(**options)
  File "/usr/local/lib/python2.7/dist-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 163, in handle_noargs
    collected = self.collect()
  File "/usr/local/lib/python2.7/dist-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 113, in collect
    handler(path, …

我正在尝试使用django-storages,boto和sorl-thumbnail将我的图像缩略图并存储在s3上.我有它工作,但它很慢,即使是小图像.当我保存表单并将图像上传到s3时,我不介意它很慢,但我希望在此之后快速显示图像.

这个SO问题的答案解释了在首次访问之前不会创建缩略图,但您可以使用get_thumbnail()预先创建它.

Django + S3(boto)+ Sorl缩略图:优化建议

我正在这样做,现在似乎所有进入thumbnail_kvstore表的条目都是在上传图像时创建的,而不是在显示时创建的.

问题是显示图像的页面仍然很慢.查看调试工具栏中的日志记录面板,看起来仍然有很多与s3的通信.看起来在上传和缓存图像和缩略图之后,页面应该快速渲染而不与s3通信.

我究竟做错了什么？谢谢!

更新:弱黑客似乎已经使它工作,但我很想知道如何正确地做到这一点:

https://github.com/asciitaxi/sorl-thumbnail/commit/545cce3f5e719a91dd9cc21d78bb973b2211bbbf

更新:@sorl的更多信息

我正在使用2个视图:

添加视图:在此视图中,我提交表单以创建包含图像的模型.图像上传到s3.在post_save信号中,我调用get_thumbnail()在需要之前生成缩略图:

im = get_thumbnail(instance.image, '360x360')

显示视图:在此视图中,我显示添加视图中生成的缩略图:

    {% thumbnail object.image "360x360" as im %}
    <img src="{{ im.url }}" width="{{ im.width }}" height="{{ im.height }}">
    {% endthumbnail %}

没有补丁:

添加视图:在kvstore表中创建3个条目,访问缓存10次(6组,4个获取),调试工具栏的日志选项卡显示"建立HTTP连接"12次

显示视图:kvstore表中只有3个条目,只有1个来自缓存,但是调试工具栏仍然说"建立HTTP连接"3次

只有第122行的更改:

添加视图:与上面相同,除了日志记录只显示"建立HTTP连接"2次DISPLAY VIEW:与上面相同,除了日志记录只显示"建立HTTP连接"1次

还在第118行添加更改:

添加视图:与上面相同,但现在我们只有2个"建立HTTP连接"消息DISPLAY VIEW:与上面相同,根本没有记录消息

更新:看起来像storage._setup()被调用两次,而storage.url()被调用一次.根据时间,我会说每个人都与s3建立联系:

1304711315.4
_setup
1304711317.84
1304711317.84
_setup
1304711320.3
1304711320.39
_url
1304711323.66

这似乎反映在boto日志中,它说"建立HTTP连接"3次.

我已经遵循了为django- storage 提供的非常简洁的指南,从本地文件存储过渡,并且遇到了这个异常:

无法加载Boto的S3绑定.

settings.py

DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
AWS_ACCESS_KEY_ID = "xxxxxx"
AWS_SECRET_ACCESS_KEY = "xxxxxxxxx"
AWS_STORAGE_BUCKET_NAME = "images"

models.py

class CameraImage(models.Model):
...
image = models.ImageField(upload_to='images')#get_image_path)

那个例外是什么意思？我如何解决它？

我正在尝试将django应用程序从django 1.6.6升级到1.7并使用python 2.7.8.当我运行时python manage.py makemigrations,我收到以下错误:

ValueError: Cannot serialize: <storages.backends.s3boto.S3BotoStorage object at 0x11116eed0>
There are some values Django cannot serialize into migration files.

以下是相关代码:

protected_storage = storages.backends.s3boto.S3BotoStorage(
      acl='private',
      querystring_auth=True,
      querystring_expire=3600,
    )


    class Document(models.Model):
        ...
        file = models.FileField(upload_to='media/docs/', max_length=10000, storage=protected_storage)

        def __unicode__(self):
            return "%s" % self.candidate

        def get_absolute_url(self):
            return reverse('documents', args=[str(self.pk)])

我已经阅读了迁移文档,并在这里阅读了类似的问题,但我一直无法解决这个问题.我的应用程序使用django-storages和boto将文件保存到Amazon S3上.任何帮助表示赞赏.

在settings.py我有:

STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'

DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
AWS_ACCESS_KEY_ID = 'xxxxxxxxxxxxx'
AWS_SECRET_ACCESS_KEY = 'xxxxxxxxxxxxx'
AWS_STORAGE_BUCKET_NAME = 'static.mysite.com'

这指向我的S3存储桶static.mysite.com并且在我这样做时工作正常manage.py collectstatic,它将所有静态文件上传到我的存储桶.不过,我还有一个水桶,我用不同的目的,想如果我有这样的模型在网站的某些地区使用,例如:

class Image(models.Model):
    myobject = models.ImageField(upload_to='my/folder')

现在Image.save()调用它时,它仍会将文件上传到S3存储桶中AWS_STORAGE_BUCKET_NAME,但是我希望它Image.save()指向另一个S3存储桶.任何清洁这样的方式？我不想改变settings.py运行时间,也不想实现任何违反django关键原则的做法,即具有可插入的易于更改的后端存储.

根据本指南和这些[1] [2]帖子,我尝试使用django-storages Boto在AWS S3上设置静态存储.

执行时collectstatic,命令成功收集STATIC_ROOT.但是,文件未上传到S3压缩,并且服务器无法提供服务.返回500错误.看日志:

错误信息:

UncompressableFileError: 'https://<myapp>.s3.amazonaws.com/static/oscar/css/styles.css' could not be found in the COMPRESS_ROOT '/var/www/<myappname>/static' or with staticfiles.

编辑:

我还将STATIC_URL更改为http://%s/ % AWS_S3_CUSTOM_DOMAIN,并且我得到了相同的错误,除了它仍在搜索https,但COMPRESS_URL是http.

UncompressableFileError: 'https://<myappname>.s3.amazonaws.com/static/oscar/css/styles.css' isn't accessible via COMPRESS_URL ('http://<myappname>.s3.amazonaws.com/') and can't be compressed

这与压缩机和Boto不兼容吗？

相关代码:

# settings/prod.py

AWS_ACCESS_KEY_ID = <Key_ID>
AWS_SECRET_ACCESS_KEY = <Secret_Key>
AWS_STORAGE_BUCKET_NAME = "<my_bucket_name>"
AWS_S3_CUSTOM_DOMAIN = "%s.s3.amazonaws.com" % AWS_STORAGE_BUCKET_NAME
STATIC_URL = "https://%s/" % AWS_S3_CUSTOM_DOMAIN
AWS_LOCATION = 'static'

DEFAULT_FILE_STORAGE = "storages.backends.s3boto.S3BotoStorage"
STATICFILES_STORAGE = "myapp.storage.s3utils.CachedS3BotoStorage" …

我希望能够将上传的图像调整为各种尺寸类别:

• 原版的
• 中(500kb)
• 小(200kb)

并将其保存到AWS S3.后来才能访问它.一种策略是将其保存在filename_small.jpg,filename_medium.jpg中,有一个辅助函数,可以附加_small,_medium来访问这些文件.我不知道如何保存所有不同的文件(调整大小),然后使用帮助程序访问它.

https://gitlab.com/firdausmah/railercom/blob/master/railercomapp/storage_backends.py

class MediaStorage(S3Boto3Storage):
    location = 'media'
    file_overwrite = False

https://gitlab.com/firdausmah/railercom/blob/master/railercomapp/models.py

class Employee(models.Model):
...
    face_image = models.FileField(upload_to=upload_to('employee/face_image/'), blank=True, storage=MediaStorage())

https://gitlab.com/firdausmah/railercom/blob/master/railercomapp/api.py

@api_view(['POST'])
def update_employee_image(request):
...
    employee = Employee.objects.get(id = employee_id)
    employee.face_image = face_image_obj
    employee.save()

我正在使用django-storages和S3Boto3Storage.我的完整工作项目是在git链接中.

我使用的是s3boto后端,而不是s3后端.

在django- storages 文档中,它表示要在settings.py文件中指定AWS_HEADERS变量:

AWS_HEADERS(可选)

如果您要设置随存储的每个文件发送的标头:

# see http://developer.yahoo.com/performance/rules.html#expires
AWS_HEADERS = {
'Expires': 'Thu, 15 Apr 2010 20:00:00 GMT',
'Cache-Control': 'max-age=86400',
}

这不适合我.

这是我的模型:

class Photo(models.Model):
    """
        docstring for Photo
        represents a single photo.. a photo can have many things associated to it like
        a project, a portfolio, etc...
    """

    def image_upload_to(instance, filename):
        today = datetime.datetime.today()
        return 'user_uploads/%s/%s/%s/%s/%s/%s/original/%s' % (instance.owner.username, today.year, today.month, today.day, today.hour, today.minute, filename)

    def thumb_upload_to(instance, filename):
        today = datetime.datetime.today()
        return 'user_uploads/%s/%s/%s/%s/%s/%s/thumb/%s' % (instance.owner.username, today.year, today.month, …