如文档所示,@shared_task装饰器允许您创建任务而无需任何具体的应用程序实例.给出的示例显示了如何装饰基于函数的任务.
如何装饰基于类的任务？
这不再适用了,scrapy的API已经改变了.
现在文档的功能是" 从脚本运行Scrapy "但我得到ReactorNotRestartable错误.
我的任务:
from celery import Task
from twisted.internet import reactor
from scrapy.crawler import Crawler
from scrapy import log, signals
from scrapy.utils.project import get_project_settings
from .spiders import MySpider
class MyTask(Task):
def run(self, *args, **kwargs):
spider = MySpider
settings = get_project_settings()
crawler = Crawler(settings)
crawler.signals.connect(reactor.stop, signal=signals.spider_closed)
crawler.configure()
crawler.crawl(spider)
crawler.start()
log.start()
reactor.run()
我对如何处理特定情况下的交易感到有点困惑.
我有一些代码可以归结为:
from django.db import transaction
@transaction.commit_on_success
def process_post():
#do stuff with database
for reply in post_replies:
process_post_reply(reply)
@transaction.commit_on_success
def process_post_reply(reply):
#do stuff with database
我想知道如果process_post_reply()失败会发生什么.
commit_on_success如何处理嵌套？是否理解提交每个process_post_reply()或如果一个失败整个process_post()回滚？
鉴于以下代码,我想知道如何填充RecordsResource每个真实的记录数据:
models.py
class Record(models.Model):
content_type = models.ForeignKey(ContentType, editable=False, null=True)
user = models.ForeignKey(User, related_name='records')
issued = models.DateTimeField(auto_now_add=True)
date = models.DateField()
def save(self, *args, **kwargs):
if not self.content_type:
self.content_type = ContentType.objects.get_for_model(self.__class__)
super(Record, self).save(*args, **kwargs)
def as_leaf_class(self):
model = self.content_type.model_class()
if model == self.__class__:
return self
return model.objects.get(pk=self.id)
class Record1(Record):
# some fields
# ...
class RecordN(Record):
# some fields
api.py
class BaseModelResource(ModelResource):
class Meta(object):
authentication = ApiKeyPlusWebAuthentication()
authorization= Authorization()
cache = SimpleCache()
throttle = CacheDBThrottle(
throttle_at=350,
# 1 day
expiration=86400 …