如何同步新的ActiveStorage镜像？

Question

从ActiveStorage开始，您可以知道定义用于存储文件的镜像。

local:
  service: Disk
  root: <%= Rails.root.join("storage") %>

amazon:
  service: S3
  access_key_id: <%= Rails.application.credentials.dig(:aws, :access_key_id) %>
  secret_access_key: <%= Rails.application.credentials.dig(:aws, :secret_access_key) %>
  region: us-east-1
  bucket: mybucket

mirror:
  service: Mirror
  primary: local
  mirrors:
    - amazon
    - another_mirror

如果在某个时间点之后添加镜像，则必须注意将所有文件复制，例如从“本地”复制到“ amazon”或“ another_mirror”。

1. 是否有一种方便的方法来保持文件同步？
2. 还是方法运行验证来检查每个服务上是否所有文件都可用？

Answer 1

我有几种可能适合您的解决方案：

遍历您的ActiveStorage Blob：

ActiveStorage::Blob.all.each do |blob|
  # work with blob
end

然后...

1. 导轨<= 6.0

   您将需要Blob的密钥，校验和和磁盘上的本地文件。

   local_file = ActiveStorage::Blob.service.primary.path_for(blob.key)
   
   # I'm picking the first mirror but you can add some code to select a specific mirror
   mirror = blob.service.mirrors.first
   
   mirror.upload(blob.key, File.open(local_file), checksum: blob.checksum)
   

2. Rails 6.1.alpha +

   只需在每个Blob上调用此按钮即可...

   blob.mirror_later
   

如果镜像中已经存在文件，您可能还想避免上传该文件。您可以这样做：

mirror = blob.service.mirrors.first

# If the file doesn't exist on the mirror, upload it
unless mirror.exist? blob.key
  # Upload file to mirror
end

放在一起，rake任务可能类似于：

# lib/tasks/active_storage.rake

namespace :active_storage do

  desc 'Ensures all files are mirrored'
  task mirror_all: [:environment] do

    # Iterate through each blob
    ActiveStorage::Blob.all.each do |blob|

      # We assume the primary storage is local
      local_file = ActiveStorage::Blob.service.primary.path_for(blob.key)

      # Iterate through each mirror
      blob.service.mirrors.each do |mirror|

        # If the file doesn't exist on the mirror, upload it
        mirror.upload(blob.key, File.open(local_file), checksum: blob.checksum) unless mirror.exist? blob.key

      end
    end
  end
end

Answer 2

(03-11-2021) On Rails > 6.1.4.1，使用active_storage > 6.1.4.1并在以下范围内：

宝石文件：

gem 'azure-storage-blob', github: 'Azure/azure-storage-ruby'

配置/环境/生产.rb

 # Store uploaded files on the local file system (see config/storage.yml for options).
  config.active_storage.service = :mirror #:microsoft or #:amazon

配置/存储.yml：

amazon:
  service: S3
  access_key_id: XXX
  secret_access_key: XXX
  region: XXX
  bucket: XXX

microsoft:
  service: AzureStorage
  storage_account_name: YYY
  storage_access_key: YYY
  container: YYY

mirror:
  service: Mirror
  primary: amazon
  mirrors: [ microsoft ]

这不起作用：

ActiveStorage::Blob.all.each do |blob|
  blob.mirror_later
end && puts("Mirroring done!")

所做的工作是：

ActiveStorage::Blob.all.each do |blob|
  ActiveStorage::Blob.service.try(:mirror, blob.key, checksum: blob.checksum)
end && puts("Mirroring done!")

不知道为什么会这样，也许 Rails 的未来版本支持它，或者它需要额外的后台作业设置，或者它最终会发生（这对我来说从未发生过）。

长话短说

如果您需要立即对整个存储进行镜像，请添加此 rake 任务并使用以下命令在给定环境中执行它bundle exec rails active_storage:mirror_all：

lib/tasks/active_storage.rake

namespace :active_storage do
  desc 'Ensures all files are mirrored'
  task mirror_all: [:environment] do
    ActiveStorage::Blob.all.each do |blob|
      ActiveStorage::Blob.service.try(:mirror, blob.key, checksum: blob.checksum)
    end && puts("Mirroring done!")
  end
end

可选：
镜像所有 blob 后，如果您希望它们实际上从正确的存储中获得服务，您可能需要更改它们的所有服务名称：

namespace :active_storage do
  desc 'Change each blob service name to microsoft'
    task switch_to_microsoft: [:environment] do
      ActiveStorage::Blob.all.each do |blob|
        blob.service_name = 'microsoft'
        blob.save
    end && puts("All blobs will now be served from microsoft!")
  end
end

config.active_storage.service=最后，在Production.rb中更改：或将主镜像设为您希望将来上传的镜像。