App Engine Bulkloader如何禁用"由于错误而退回"

Question

我必须使用App Engine批量加载程序上传大量实体(800k).一切都已建立,事情进展顺利,除了我得到一个导致这种情况发生的神秘错误:

[WorkerThread-0] Backing off due to errors: 1.0 seconds
[WorkerThread-0] Backing off due to errors: 2.0 seconds
[WorkerThread-0] Backing off due to errors: 4.0 seconds
[WorkerThread-0] Backing off due to errors: 8.0 seconds

这很好,直到我让它全天运行:

[WorkerThread-0] Backing off due to errors: 2048.0 seconds
[WorkerThread-0] Backing off due to errors: 4096.0 seconds

我怎么能禁用它？它使我的上传永远.从字面上看,每千个实体就会发生这种情况,所以我可能会等待2^sum(1 to 800)几秒钟.这显然不太理想.

我不知道错误是什么,除了它说我有"错误",但似乎上传就好了,如果我停止并每小时恢复它.

Answer 1

我修改google_appengine/google/appengine/tools/adaptive_thread_pool.py为忽略self.__backing_off变量.具体来说,我将其重置为1.0第426行.

这是多么荒谬的Google设计......