在theading.Thread中使用sqlalchemy scoped_session

Question

我在使用sqlalchemy和线程时遇到了问题.

import queue
import threading

import sqlalchemy
from sqlalchemy import create_engine, Column, Integer, String, Sequence
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
from sqlalchemy.orm.scoping import scoped_session

engine = create_engine('sqlite:///:memory:', echo=False)
Base = declarative_base()

class User(Base):
    __tablename__ = 'users'
    id = Column(Integer, Sequence('user_id_seq'), primary_key=True)
    name = Column(String)
    fullname = Column(String)
    password = Column(String)

    def __repr__(self):
        return "<User(name='%s', fullname='%s', password='%s')>" % (
        self.name, self.fullname, self.password)
Base.metadata.create_all(engine)

sessionfactory = sessionmaker(bind=engine)

# called by each thread
def write_name(q, name, sessionfactory):
    session = scoped_session(sessionfactory)
    ed_user = User(name=name, fullname='Power', password='edspassword')
    session.add(ed_user)
    session.commit()
    q.put(name)

names = ["Max", "Austin"]

q = queue.Queue()

for u in names:
    t = threading.Thread(target=write_name, args = (q, u, sessionfactory))
    t.daemon = True
    t.start()

s = q.get()

这导致:

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: users [SQL: 'INSERT INTO users (name, fullname, password) VALUES (?, ?, ?)'] [parameters: ('Max', 'Power', 'edspassword')]

但它可以在主线程中添加和读取数据.此外,我假设我需要在多进程上使用线程,因为scoped_session使用线程本地存储.

Answer 1

主要问题是您不能与仅存在于内存中的SQLite数据库建立多个连接,因为每个连接都将创建一个新的空数据库.请参阅SQLAlchemy文档.简而言之,您需要像这样创建引擎,以确保只有一个可以跨线程共享的实例.

from sqlalchemy.pool import StaticPool
engine = create_engine('sqlite://:memory:',
    connect_args={'check_same_thread': False},
    poolclass=StaticPool, echo=True)

一旦你这样做,你就不需要了scoped_session,因为关键scoped_session是要为每个线程创建一个连接,而你在这里特别不能这样做.

另外,请注意,scoped_session如果您希望它正常工作(使用非SQLite引擎),您应该只有一个实例.您应该将其视为全局变量,然后它将能够处理线程本地的东西.