我们正在运行芹菜3.1.8,Redis作为经纪人后端.我们的配置很简单,我们不关心结果,为此我们设置
CELERY_IGNORE_RESULT = True
但是我们注意到我们的redis代理实例占用了大量内存(~3GB),而其中的队列和消息的实际数量在任何给定时刻都非常小.在检查了RDB文件之后,我们注意到有一些巨大的列表:
database,type,key,size_in_bytes,encoding,num_elements,len_largest_element
0,list,"ed2d26b5-b8fb-3478-ace7-6714e8a7b4ed.reply.celery.pidbox",70733024,linkedlist,4,17696745
0,list,"e3eda502-27eb-348d-a86e-1d78fc31b165.reply.celery.pidbox",35350876,linkedlist,2,17686857
0,list,"6421b8c1-bbc0-3a59-a7ee-f26450552a60.reply.celery.pidbox",35443327,linkedlist,2,17762611
0,list,"8439dcd0-921e-3922-8504-9057b6c9834a.reply.celery.pidbox",106088780,linkedlist,6,17696745
0,list,"85c667c8-63b6-338f-b00a-e1f2cd4da143.reply.celery.pidbox",17762845,linkedlist,1,17762611
0,list,"05d0c0f2-9530-37f3-a9cb-189fc237303c.reply.celery.pidbox",106088769,linkedlist,6,17696743
0,list,"d3f200fd-c81e-3d6d-acf3-d0e9021e7e5c.reply.celery.pidbox",35431615,linkedlist,2,17762611
0,list,"7b4291c7-b916-3806-910b-c250c9a7fece.reply.celery.pidbox",88401866,linkedlist,5,17696745
0,list,"0c8b64c1-7efe-3070-b2e3-980f395b84e8.reply.celery.pidbox",123752294,linkedlist,7,17696743
0,list,"e5cf288b-8ced-3f6c-891e-34e2d302c89c.reply.celery.pidbox",70711492,linkedlist,4,17691717
0,list,"a9cafe29-204d-3d97-9b7b-322a847d0789.reply.celery.pidbox",53121613,linkedlist,3,17762611
0,list,"1c1f90ca-1fe1-35e4-a144-3a97177a674b.reply.celery.pidbox",35431683,linkedlist,2,17762611
这些列表包含一些json项目,每个项目包含一个巨大的Body.
即使我们设置芹菜忽略结果,为什么这些回复pidbox出现的指针？如果我们对任务的结果不感兴趣,它们是否需要芹菜？我们想摆脱这些,因为他们正在转变一个应该需要大约10mb内存才能占用~3gb内存的实例.
任何帮助都会很棒,如果您需要有关配置的更多信息,请告诉我.