如何从云功能连接Google Cloud SQL？

Question

我正在尝试使用Cloud Functions for Firebase来构建一个与Google Cloud SQL(PostgreSQL)实例对话的API.

我正在使用HTTP(S)触发器.

当我列出桌面的IP地址时,我可以使用本地计算机上的函数node.js代码连接到Cloud SQL.但是当我部署时,我无法连接,我无法弄清楚Firebase功能服务器的HOST IP地址到白名单.

您如何通过Cloud Functions for Firebase与Google Cloud SQL对话？

谢谢!

// Code Sample, of what's working on Localhost.
var functions = require('firebase-functions');

var pg = require('pg');
var pgConfig = {
  user: functions.config().pg.user,
  database: functions.config().pg.database,
  password: functions.config().pg.password,
  host: functions.config().pg.host
}

exports.helloSql = functions.https.onRequest((request, response) => {
  console.log('connecting...');
  try {
    client.connect(function(err) {
      if (err) throw err;

      console.log('connection success');
      console.log('querying...');

      client.query('SELECT * FROM guestbook;', function(err, result){
        if (err) throw err;

        console.log('querying success.');
        console.log('Results: ', result);
        console.log('Ending...');

        client.end(function(err){
          if (err) throw err;
          console.log('End success.');
          response.send(result);
        });
      });

    });
  } catch(er) {
    console.error(er.stack)
    response.status(500).send(er);
  }
});

Answer 1

我在#36388165的进一步讨论中找到了答案.

免责声明:这似乎没有正式公布,因此可能会在之后发生变化.我也只在mysql中测试.但是这个解决方案的性质,我认为应该像pg模块一样工作(它似乎接受域套接字路径作为主机参数)

编辑(2017/12/7):谷歌似乎提供官方早期访问,同样的方法仍然有效.
编辑(2018/07/04):似乎有人只是复制并粘贴我的示例代码并遇到麻烦.正如谷歌所说,你应该使用连接池来避免sql连接泄漏.(它导致ECONNREFUSE)所以我稍微更改了示例代码.

在https://issuetracker.google.com/issues/36388165#comment44谷歌家伙说云功能实例可以通过特殊路径中的域套接字与云sql交谈'/ cloudsql/$ PROJECT_ID:$ REGION:$ DBNAME'.

我实际上可以从下面的云功能代码连接和操作云SQL.

const mysql = require('mysql');
const pool = mysql.createPool({
    connectionLimit : 1,
    socketPath: '/cloudsql/' + '$PROJECT_ID:$REGION:$SPANNER_INSTANCE_NAME',
    user: '$USER',
    password: '$PASS',
    database: '$DATABASE'
});
exports.handler = function handler(req, res) {
    //using pool instead of creating connection with function call
    pool.query(`SELECT * FROM table where id = ?`, 
                                req.body.id, function (e, results) {
        //made reply here
    });
};

我希望这对那些迫不及待谷歌正式宣布的人有所帮助.

Answer 2

目前还不可能.然而,它是问题跟踪器#36388165上的功能请求:

目前不支持从Cloud Functions连接到Cloud SQL,因为UNIX套接字不存在(导致ENOENT)并且没有已定义的IP范围进行白名单(导致ETIMEDOUT).一种可能性是从云SQL实例中将0.0.0.0/0列入白名单,但出于安全原因,不建议这样做.

如果这是一个重要的功能,我建议您访问issuetracker并为功能请求加注星标,以帮助它获得普及.

Answer 3

在GCP> SQL> Instances页面上查找数据库区域和实例名称:

运行以下命令将数据库密码保存到Firebase环境:

$ firebase functions:config:set \
    db.user="<username>" \
    db.password="<password>" \
    db.database="<database>"

然后...

db.js

const { Pool } = require('pg');
const { config } = require('firebase-functions');

const project = process.env.GCP_PROJECT;
const region = 'europe-west1';
const instance = 'db';

module.exports = new Pool({
  max: 1,
  host: `/cloudsql/${project}:${region}:${instance}`,
  ...config().db
});

someFunction.js

const { https } = require('firebase-functions');
const db = require('./db');

module.exports = https.onRequest((req, res) =>
  db
    .query('SELECT version()')
    .then(({ rows: [{ version }]) => {
      res.send(version);
    }));

另请参阅 /sf/answers/3417752621/(通过Babel使用现代JavaScript语法)

Answer 4

现在有关于此的官方文档,截至2018年7月仍处于测试阶段

https://cloud.google.com/functions/docs/sql

Answer 5

使用 TCP 和 UNIX 域套接字从 GOOGLE 云函数连接到云 SQL 2020

1.创建一个新项目

gcloud projects create gcf-to-sql
gcloud config set project gcf-to-sql
gcloud projects describe gcf-to-sql

2.为您的项目启用计费：https://cloud.google.com/billing/docs/how-to/modify-project

3.设置计算项目信息元数据：

gcloud compute project-info describe --project gcf-to-sql
#Enable the Api, and you can check that default-region,google-compute-default-zone are not set. Set the metadata.
gcloud compute project-info add-metadata --metadata google-compute-default-region=europe-west2,google-compute-default-zone=europe-west2-b

4.启用服务网络API：

gcloud services list --available
gcloud services enable servicenetworking.googleapis.com

5.创建2个云sql实例，（一个具有内部IP，一个具有公共IP）- https://cloud.google.com/sql/docs/mysql/create-instance：

6.a具有外部ip的Cloud Sql实例：

#Create the sql instance in the 
gcloud --project=con-ae-to-sql beta sql instances create database-external --region=europe-west2
#Set the password for the "root@%" MySQL user:
gcloud sql users set-password root --host=% --instance database-external --password root 
#Create a user
gcloud sql users create user_name --host=% --instance=database-external  --password=user_password
#Create a database
gcloud sql databases create user_database --instance=database-external
gcloud sql databases list --instance=database-external

6.b 具有内部ip的Cloud Sql实例：

i.#Create a private connection to Google so that the VM instances in the default VPC network can use private services access to reach Google services that support it.

gcloud compute addresses create google-managed-services-my-network     --global  --purpose=VPC_PEERING --prefix-length=16  --description="peering range for Google"  --network=default --project=con-ae-to-sql
gcloud services vpc-peerings connect --service=servicenetworking.googleapis.com --ranges=google-managed-services-my-network  --network=default  --project=con-ae-to-sql
#Check whether the operation was successful.
gcloud services vpc-peerings operations describe     --name=operations/pssn.dacc3510-ebc6-40bd-a07b-8c79c1f4fa9a
#Listing private connections
gcloud services vpc-peerings list --network=default --project=con-ae-to-sql
 
ii.Create the instance:

gcloud --project=con-ae-to-sql beta sql instances create database-ipinternal --network=default --no-assign-ip  --region=europe-west2
#Set the password for the "root@%" MySQL user:
gcloud sql users set-password root --host=% --instance database-ipinternal --password root
#Create a user
gcloud sql users create user_name --host=% --instance=database-ipinternal  --password=user_password
#Create a database
gcloud sql databases create user_database --instance=database-ipinternal
gcloud sql databases list --instance=database-ipinternal 


gcloud sql instances list
gcloud sql instances describe database-external
gcloud sql instances describe database-ipinternal
#Remember the instances connectionName

好的，我们有两个 mysql 实例，我们将使用无服务器访问和 TCP 从 Google Cloud Functions 连接到数据库 ipinternal，并使用 unix 域套接字从 Google Cloud Functions 连接到数据库外部。

7.启用 Cloud SQL 管理 API

gcloud services list --available
gcloud services enable sqladmin.googleapis.com

注意：默认情况下，Cloud Functions 不支持使用 TCP 连接到 Cloud SQL 实例。除非您配置了无服务器 VPC 访问，否则您的代码不应尝试使用 IP 地址（例如 127.0.0.1 或 172.17.0.1）访问实例。

8.a 确保为您的项目启用无服务器 VPC 访问 API：

gcloud services enable vpcaccess.googleapis.com

8.b 创建连接器：

gcloud compute networks vpc-access connectors create serverless-connector --network default --region europe-west2 --range 10.10.0.0/28
#Verify that your connector is in the READY state before using it
gcloud compute networks vpc-access connectors describe serverless-connector --region europe-west2

9.为您的云功能创建一个服务帐户。确保您的服务的服务帐户具有以下 IAM 角色：Cloud SQL 客户端，并且为了通过内部 IP 从 App Engine Standard 连接到 Cloud Sql，我们还需要角色计算网络用户。

gcloud iam service-accounts create cloud-function-to-sql
gcloud projects add-iam-policy-binding gcf-to-sql --member serviceAccount:cloud-function-to-sql@gcf-to-sql.iam.gserviceaccount.com   --role roles/cloudsql.client
gcloud projects add-iam-policy-binding gcf-to-sql --member serviceAccount:cloud-function-to-sql@gcf-to-sql.iam.gserviceaccount.com  --role roles/compute.networkUser

现在我已经配置好了

1. 使用 Tcp 和 unix domanin 套接字从 Google Cloud Functions 连接到 Cloud Sql

cd app-engine-standard/
ls
#main.py requirements.txt

cat requirements.txt
sqlalchemy
pymysql
      
cat main.py 
import pymysql
from sqlalchemy import create_engine


 def gcf_to_sql(request):

    engine_tcp = create_engine('mysql+pymysql://user_name:user_password@10.36.0.3:3306')
    existing_databases_tcp = engine_tcp.execute("SHOW DATABASES;")
    con_tcp = "Connecting from Google Cloud Functions to Cloud SQL using TCP: databases => " + str([d[0] for d in existing_databases_tcp]).strip('[]') + "\n"
    engine_unix_socket = create_engine('mysql+pymysql://user_name:user_password@/user_database?unix_socket=/cloudsql/gcf-to-sql:europe-west2:database-external')
    existing_databases_unix_socket = engine_unix_socket.execute("SHOW DATABASES;")
    con_unix_socket = "Connecting from Google Cloud Function  to Cloud SQL using Unix Sockets: tables in sys database:  => " + str([d[0] for d in existing_databases_unix_socket]).strip('[]') + "\n"
    return con_tcp + con_unix_socket
     

2.部署云功能：

gcloud beta functions deploy gcf_to_sql --runtime python37 --region europe-west2 --vpc-connector projects/gcf-to-sql/locations/europe-west2/connectors/serverless-connector  --trigger-http
 

3.进入云功能，选择gcf-to-sql、测试，测试功能：

#Connecting from Google Cloud Functions to Cloud SQL using TCP: databases => 'information_schema', 'mysql', 'performance_schema', 'sys', 'user_database'
#Connecting from Google Cloud Function  to Cloud SQL using Unix Sockets: tables in sys database:  => 'information_schema', 'mysql', 'performance_schema', 'sys', 'user_database'

成功！