小编Tib*_*ibo的帖子

我在python中遇到这个问题:

• 我有一个URL队列,我需要不时检查
• 如果队列已满,我需要处理队列中的每个项目
• 队列中的每个项目必须由单个进程处理(多处理)

到目前为止,我设法实现这个"手动"这样:

while 1:
        self.updateQueue()

        while not self.mainUrlQueue.empty():
            domain = self.mainUrlQueue.get()

            # if we didn't launched any process yet, we need to do so
            if len(self.jobs) < maxprocess:
                self.startJob(domain)
                #time.sleep(1)
            else:
                # If we already have process started we need to clear the old process in our pool and start new ones
                jobdone = 0

                # We circle through each of the process, until we find one free ; only then leave the loop 
                while jobdone == …