小编Vla*_*nko的帖子

如何将DataFrame的Spark sql表达式中的空值写入数据库表?(IllegalArgumentException:无法获取null的JDBC类型)

java.lang.IllegalArgumentException: Can't get JDBC type for null尝试运行以下示例时收到错误:

    ...
    val spark = SparkSession.builder
    .master("local[*]")
    .appName("Demo")
    .detOrCreate()

    import spark.implicits._

    //load first table
    val df_one = spark.read
    .format("jdbc")
    .option("url",myDbUrl)
    .option("dbtable",myTableOne)
    .option("user",myUser)
    .option("password",myPassw)
    .load()

    df_one.createGlobalTempView("table_one")

    //load second table
    val df_two = spark.read
    .format("jdbc")
    .option("url",myUrl)
    .option("dbtable",myTableTwo)
    .option("user",myUser)
    .option("password",myPassw)
    .load()

    df_two.createGlobalTempView("table_two")

    //perform join of two tables
    val df_result = spark.sql(
    "select o.field_one, t.field_two, null as field_three "+
    " from global_temp.table_one o, global_temp.table_two t where o.key_one = t.key_two"
    )

//Error there:
    df_result.write
    .format(jdbc)
    .option("dbtable",myResultTable)
    .option("url",myDbUrl)
    .option("user",myUser) …
Run Code Online (Sandbox Code Playgroud)

apache-spark apache-spark-sql

7
推荐指数
1
解决办法
2784
查看次数

标签 统计

apache-spark ×1

apache-spark-sql ×1