BigQuery StandardSQL udf调用外部.jar库

Question

如何在bigquery UDF中调用外部java库.可能吗？

以下是我的用例:是否可以在标准sql bigquery UDF中进行lat/long计算.

import org.gavaghan.geodesy.*;

...

GeodeticCalculator geoCalc = new GeodeticCalculator();

Ellipsoid reference = Ellipsoid.WGS84;  

GlobalPosition pointA = new GlobalPosition(latitude, longitude, 0.0); // Point A

GlobalPosition userPos = new GlobalPosition(userLat, userLon, 0.0); // Point B

double distance = geoCalc.calculateGeodeticCurve(reference, userPos, pointA).getEllipsoidalDistance(); 

Answer 1

您可以尝试使用BigQuery GIS(公共alpha)而不是UDF方向

根据您的示例 - 它可能看起来像下面简化/愚蠢的例子

#standardSQL
WITH start AS (
    SELECT 
        state_name state,
        CAST(internal_point_lat AS FLOAT64) lat, 
        CAST(internal_point_lon AS FLOAT64) lon
    FROM `bigquery-public-data.utility_us.us_states_area`
    WHERE state_name IN ('California', 'Arizona')
),  finish AS (
    SELECT 
        state_name state,
        CAST(internal_point_lat AS FLOAT64) lat, 
        CAST(internal_point_lon AS FLOAT64) lon
    FROM `bigquery-public-data.utility_us.us_states_area`
    WHERE state_name IN ('Florida', 'New York')
)
SELECT 
    start.state start,
    finish.state finish,
    ROUND(ST_DISTANCE(ST_GEOGPOINT(start.lon,start.lat),ST_GEOGPOINT(finish.lon,finish.lat))*0.000621371,2) distance
FROM start, finish    

Row start       finish      distance
1   California  Florida     2223.17
2   California  New York    2331.66
3   Arizona     Florida     1761.82
4   Arizona     New York    2019.12

注意:BQ-GIS处于公共alpha状态,因此您的GCP项目需要列入白名单才能尝试这些查询.您可以在此处注册以请求访问权限.

更新(根据评论)

我知道了.所以GIS示例只是"偶然"的例子而不是问题的焦点.得到它了.从UDF开始 - 现在BigQuery仅支持JavaScript和SQL UDF,并且支持JS UDF外部库.到目前为止,java/python运气不佳.您可以检查的唯一其他疯狂选项是使用Web程序集.
您可以查看几个示例来获得一个想法:

• SQL和JS之外的BigQuery:大规模运行C和Rust代码
• 使用web程序集从bigquery调用c函数