使用BufferedWriter in flask whooshalchemy

Question

嗨,我正在运行带有postgreSQL数据库的烧瓶应用程序.使用多个worker时,我得到LockErrors.我了解到这是因为嗖嗖搜索会锁定数据库

http://stackoverflow.com/questions/36632787/postgres-lockerror-how-to-investigate

正如在这个链接中所解释的,我必须使用BufferedWriter ...我谷歌周围,但我真的无法弄清楚如何实现它？这是我的数据库设置方面的嗖

import sys
if sys.version_info >= (3, 0):
    enable_search = False
else:
    enable_search = True
    import flask.ext.whooshalchemy as whooshalchemy

class User(db.Model):
    __searchable__ = ['username','email','position','institute','id'] # these fields will be indexed by whoosh

    id = db.Column(db.Integer, primary_key=True)
    username = db.Column(db.String(100), index=True)
    ...

    def __repr__(self):
        return '<User %r>' % (self.username)

if enable_search:
    whooshalchemy.whoosh_index(app, User)

感谢carl非常感谢

编辑:如果没有能力并行访问烧瓶 - whosshsqlalchemy你有什么选择吗？

Answer 1

正如您可以在这里阅读的：

http://whoosh.readthedocs.io/en/latest/threads.html

只有一位作家可以持有锁。缓冲写入器，将您的数据保留一段时间，但是......在某些时候您的对象被存储，这意味着 - 锁定。

根据该文档，异步编写器是您正在寻找的东西，但是...它将尝试存储您的数据，如果失败 - 它将创建额外的线程，然后重试。假设您要扔掉 1000 个新物品。您最终可能会得到大约 1000 个线程。最好将每个插入视为一个任务，并将其发送到单独的线程。如果有很多进程，您可以堆叠这些任务。例如 - 插入 10，然后等待。如果这10个批量插入的话，会在短时间内吗？会工作-一段时间...

编辑

使用异步读取器的示例 - 进行缓冲 - 只需重命名导入和用法。

import os, os.path
from whoosh import index
from whoosh.fields import SchemaClass, TEXT, KEYWORD, ID

if not os.path.exists("data"):
    os.mkdir("data")

# http://whoosh.readthedocs.io/en/latest/schema.html
class MySchema(SchemaClass):
    path = ID(stored=True)
    title = TEXT(stored=True)
    icon = TEXT
    content = TEXT(stored=True)
    tags = KEYWORD

# http://whoosh.readthedocs.io/en/latest/indexing.html
ix = index.create_in("data", MySchema, indexname="myindex")

writer = ix.writer()
writer.add_document(title=u"My document", content=u"This is my document!",
                    path=u"/a", tags=u"first short", icon=u"/icons/star.png")
writer.add_document(title=u"Second try", content=u"This is the second example.",
                    path=u"/b", tags=u"second short", icon=u"/icons/sheep.png")
writer.add_document(title=u"Third time's the charm", content=u"Examples are many.",
                    path=u"/c", tags=u"short", icon=u"/icons/book.png")
writer.commit()

# needed to release lock
ix.close()

#http://whoosh.readthedocs.io/en/latest/api/writing.html#whoosh.writing.AsyncWriter
from whoosh.writing import AsyncWriter

ix = index.open_dir("data", indexname="myindex")

writer = AsyncWriter(ix)
writer.add_document(title=u"My document no 4", content=u"This is my document!",
                    path=u"/a", tags=u"four short", icon=u"/icons/star.png")
writer.add_document(title=u"5th try", content=u"This is the second example.",
                    path=u"/b", tags=u"5 short", icon=u"/icons/sheep.png")
writer.add_document(title=u"Number six is coming", content=u"Examples are many.",
                    path=u"/c", tags=u"short", icon=u"/icons/book.png")
writer.commit()