如何创建mysql连接池或任何更好的方法来初始化多个数据库？

Question

在我的代码中,我打开两个mysql连接并使用HTTP请求将数据插入数据库

g.db = mysql.connector.connect(user=a ,password=password, host=localhost,database=mysq1)
g.db1 = mysql.connector.connect(user=b,password=password, host=localhost,database=mysql2)

@app.route('/user/<db>')  
def insert(db):
   #code for inserting data into mysql1 database
   #code for inserting data into mysql2 database

我正在发出HTTP请求以选择数据库.

  curl -i 'localhost:5000/user/mysql1' #

它运行良好,数据被插入到选定的数据库中.但我正在考虑为这两个连接创建连接池,然后使用该池.

问题:

1. 如何实现mysql连接池？

2. 还有其他更好的初始化连接的方法.当前连接在每个请求时打开.

Answer 1

#!/usr/bin/python
# -*- coding: utf-8 -*-
import time
import mysql.connector.pooling


dbconfig = {
    "host":"127.0.0.1",
    "port":"3306",
    "user":"root",
    "password":"123456",
    "database":"test",
}


class MySQLPool(object):
    """
    create a pool when connect mysql, which will decrease the time spent in 
    request connection, create connection and close connection.
    """
    def __init__(self, host="172.0.0.1", port="3306", user="root",
                 password="123456", database="test", pool_name="mypool",
                 pool_size=3):
        res = {}
        self._host = host
        self._port = port
        self._user = user
        self._password = password
        self._database = database

        res["host"] = self._host
        res["port"] = self._port
        res["user"] = self._user
        res["password"] = self._password
        res["database"] = self._database
        self.dbconfig = res
        self.pool = self.create_pool(pool_name=pool_name, pool_size=pool_size)

    def create_pool(self, pool_name="mypool", pool_size=3):
        """
        Create a connection pool, after created, the request of connecting 
        MySQL could get a connection from this pool instead of request to 
        create a connection.
        :param pool_name: the name of pool, default is "mypool"
        :param pool_size: the size of pool, default is 3
        :return: connection pool
        """
        pool = mysql.connector.pooling.MySQLConnectionPool(
            pool_name=pool_name,
            pool_size=pool_size,
            pool_reset_session=True,
            **self.dbconfig)
        return pool

    def close(self, conn, cursor):
        """
        A method used to close connection of mysql.
        :param conn: 
        :param cursor: 
        :return: 
        """
        cursor.close()
        conn.close()

    def execute(self, sql, args=None, commit=False):
        """
        Execute a sql, it could be with args and with out args. The usage is 
        similar with execute() function in module pymysql.
        :param sql: sql clause
        :param args: args need by sql clause
        :param commit: whether to commit
        :return: if commit, return None, else, return result
        """
        # get connection form connection pool instead of create one.
        conn = self.pool.get_connection()
        cursor = conn.cursor()
        if args:
            cursor.execute(sql, args)
        else:
            cursor.execute(sql)
        if commit is True:
            conn.commit()
            self.close(conn, cursor)
            return None
        else:
            res = cursor.fetchall()
            self.close(conn, cursor)
            return res

    def executemany(self, sql, args, commit=False):
        """
        Execute with many args. Similar with executemany() function in pymysql.
        args should be a sequence.
        :param sql: sql clause
        :param args: args
        :param commit: commit or not.
        :return: if commit, return None, else, return result
        """
        # get connection form connection pool instead of create one.
        conn = self.pool.get_connection()
        cursor = conn.cursor()
        cursor.executemany(sql, args)
        if commit is True:
            conn.commit()
            self.close(conn, cursor)
            return None
        else:
            res = cursor.fetchall()
            self.close(conn, cursor)
            return res


if __name__ == "__main__":
    mysql_pool = MySQLPool(**dbconfig)
    sql = "select * from store WHERE create_time < '2017-06-02'"

    # test...
    while True:
        t0 = time.time()
        for i in range(10):
            mysql_pool.execute(sql)
            print i
        print "time cousumed:", time.time() - t0

您可以在create_pool()最后创建一个连接池,最终导致连接池MySQLConnectionPool(),当您需要连接到MySQL时,您可以get_connection()从池中获得连接,当您不需要连接时,您可以将连接添加回池中与conn.close().

Answer 2

使用 ORM 框架可以让事情变得更容易，下面是我们在没有任何 ORM 框架的情况下创建连接池的基本和通用方法。

1. mysql.connector.pooling模块实现池化。

2. 池在向请求者提供连接时打开许多连接并处理线程安全。

3. 连接池的大小可以在创建池时配置。此后无法调整其大小。

创建自己的池并在连接池的参数中将其命名为myPool，您还可以声明池大小= 5（这是数据库连接的数量）。

请参阅下文了解更多信息：

dbconfig = {
  "database": "test",
  "user":     "joe"
}

cnx = mysql.connector.connect(pool_name = "mypool",
                              pool_size = 3,
                              **dbconfig)

dbconfig，数据库配置是您每次更改数据库时提供所有配置详细信息的地方。事实上，如果您愿意，您可以拥有多个数据库。

请参阅此处的 MySQL 文档

我们可以看到更多关于如何声明这个参数的信息：

MySQLConnectionPool(pool_name=None,
                    pool_size=5,
                    pool_reset_session=True,
                    **kwargs)

该构造函数实例化一个管理连接池的对象。

详细论证：

1. pool_name: The pool name. If this argument is not given, Connector/Python automatically generates the name, composed from whichever of the host, port, user, and database connection arguments are given in kwargs, in that order.

It is not an error for multiple pools to have the same name. An application that must distinguish pools by their
**pool_name** property should create each pool with a distinct name.

2. pool_size: The pool size. If this argument is not given, the default is 5.

你应该在这里看到一些不错的文档

为了使连接池成为多线程，stackoverflw 上的这篇文章可能确实有帮助。请看这个帖子