迁移App Engine模型的方法

Question

数据库迁移是一种流行的模式,特别是Ruby on Rails.由于迁移指定了如何模拟旧数据以适应新模式,因此当您拥有必须快速可靠地转换的生产数据时,它们会很有用.

但是,在App Engine中迁移模型很困难,因为顺序处理所有实体很困难,并且没有脱机操作可以在一个大事务中有效地迁移所有内容.

有哪些技术可以修改db.Model"模式"并迁移数据以适应新模式？

Answer 1

这就是我的工作.

我有一个MigratingModel类,我的所有模型都继承自该类.这是migrating_model.py:

"""Models which know how to migrate themselves"""

import logging
from google.appengine.ext import db
from google.appengine.api import memcache

class MigrationError(Exception):
  """Error migrating"""

class MigratingModel(db.Model):
  """A model which knows how to migrate itself.

  Subclasses must define a class-level migration_version integer attribute.
  """

  current_migration_version = db.IntegerProperty(required=True, default=0)

  def __init__(self, *args, **kw):
    if not kw.get('_from_entity'):
      # Assume newly-created entities needn't migrate.
      try:
        kw.setdefault('current_migration_version',
                      self.__class__.migration_version)
      except AttributeError:
        msg = ('migration_version required for %s'
                % self.__class__.__name__)
        logging.critical(msg)
        raise MigrationError, msg
    super(MigratingModel, self).__init__(*args, **kw)

  @classmethod
  def from_entity(cls, *args, **kw):
    # From_entity() calls __init__() with _from_entity=True
    obj = super(MigratingModel, cls).from_entity(*args, **kw)
    return obj.migrate()

  def migrate(self):
    target_version = self.__class__.migration_version
    if self.current_migration_version < target_version:
      migrations = range(self.current_migration_version+1, target_version+1)
      for self.current_migration_version in migrations:
        method_name = 'migrate_%d' % self.current_migration_version
        logging.debug('%s migrating to %d: %s'
                       % (self.__class__.__name__,
                          self.current_migration_version, method_name))
        getattr(self, method_name)()
      db.put(self)
    return self

MigratingModel拦截从原始数据存储区实体到完整db.Model实例的转换.如果current_migration_version落后于班级的最新版本migration_version,那么它会运行一系列migrate_N()方法来完成繁重的工作.

例如:

"""Migrating model example"""

# ...imports...

class User(MigratingModel):
  migration_version = 3

  name = db.StringProperty() # deprecated: use first_name and last_name

  first_name = db.StringProperty()
  last_name = db.StringProperty()
  age = db.IntegerProperty()

  invalid = db.BooleanProperty() # to search for bad users

  def migrate_1(self):
    """Convert the unified name to dedicated first/last properties."""
    self.first_name, self.last_name = self.name.split()

  def migrate_2(self):
    """Ensure the users' names are capitalized."""
    self.first_name = self.first_name.capitalize()
    self.last_name = self.last_name.capitalize()

  def migrate_3(self):
    """Detect invalid accounts"""
    if self.age < 0 or self.age > 85:
      self.invalid = True

在繁忙的站点上,如果db.put()失败,migrate()方法应该重试,如果迁移不起作用,可能会记录严重错误.

我还没有到达那里,但在某些时候我可能会从一个单独的文件混合我的迁移.

最后的想法

在App Engine上很难测试.在测试环境中很难访问生产数据,此时很难做到连贯的快照备份.因此,对于重大更改,请考虑创建一个使用完全不同的模型名称的新版本,该名称从旧模型导入并根据需要进行迁移.(例如,User2而不是User).这样,如果您需要回退到以前的版本,您就可以有效地备份数据.