Superset 仪表板因超时错误而失败：已达到队列池大小 5 溢出 10 的限制

Question

我的 Docker Superset (Ubuntu 18.04 AWS EC2)，我的仪表板有 18 个小部件；我使用MYSQL作为后端数据库。报告从MySQL读取数据。在大多数小部件上运行仪表板时，我收到以下错误消息：

意外的错误：

TimeoutError: QueuePool limit of size 5溢出10达到，连接超时，超时30（此错误的背景位于： http: //sqlalche.me/e/13/3o7r）

详细信息 错误信息

Traceback (most recent call last): File "/app/superset/views/base.py", line 164, in wraps return f(self, *args, **kwargs) File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/security/decorators.py", line 149, in wraps permission_str, self.class_permission_name File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/security/manager.py", line 1159, in has_access if current_user.is_authenticated: File "/usr/local/lib/python3.7/site-packages/werkzeug/local.py", line 347, in __getattr__ return getattr(self._get_current_object(), name) File "/usr/local/lib/python3.7/site-packages/werkzeug/local.py", line 306, in _get_current_object return self.__local() File "/usr/local/lib/python3.7/site-packages/flask_login/utils.py", line 26, in <lambda> current_user = LocalProxy(lambda: _get_user()) File "/usr/local/lib/python3.7/site-packages/flask_login/utils.py", line 335, in _get_user current_app.login_manager._load_user() File "/usr/local/lib/python3.7/site-packages/flask_login/login_manager.py", line 359, in _load_user return self.reload_user() File "/usr/local/lib/python3.7/site-packages/flask_login/login_manager.py", line 321, in reload_user user = self.user_callback(user_id) File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/security/manager.py", line 1674, in load_user return self.get_user_by_id(int(pk)) File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/security/sqla/manager.py", line 233, in get_user_by_id return self.get_session.query(self.user_model).get(pk) File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 1018, in get return self._get_impl(ident, loading.load_on_pk_identity) File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 1135, in _get_impl return db_load_fn(self, primary_key_identity) File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 286, in load_on_pk_identity return q.one() File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3490, in one ret = self.one_or_none() File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3459, in one_or_none ret = list(self) File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3535, in __iter__ return self._execute_and_instances(context) File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3557, in _execute_and_instances querycontext, self._connection_from_session, close_with_result=True File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3572, in _get_bind_args mapper=self._bind_mapper(), clause=querycontext.statement, **kw File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3550, in _connection_from_session conn = self.session.connection(**kw) File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1141, in connection execution_options=execution_options, File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1147, in _connection_for_bind engine, execution_options File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 433, in _connection_for_bind conn = bind._contextual_connect() File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2302, in _contextual_connect self._wrap_pool_connect(self.pool.connect, None), File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2336, in _wrap_pool_connect return fn() File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 364, in connect return _ConnectionFairy._checkout(self) File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 778, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 495, in checkout rec = pool._do_get() File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/impl.py", line 132, in _do_get code="3o7r", sqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30 (Background on this error at: http://sqlalche.me/e/13/3o7r)

Answer 1

我在 superset_config.py 中添加了以下内容并增加了 MySQL 服务器 CPU 和内存。经过这些更改后，我不再收到错误消息（达到 QueuePool limit of size 5 Overflow 10）：

SQLALCHEMY_POOL_SIZE = 45

SQLALCHEMY_MAX_OVERFLOW = 30

SQLALCHEMY_POOL_TIMEOUT = 180