小编cle*_*sky的帖子

Pseudocolumn in Spark JDBC

I am using a query to fetch data from MYSQL as follows:

var df = spark.read.format("jdbc")
         .option("url", "jdbc:mysql://10.0.0.192:3306/retail_db")
         .option("driver" ,"com.mysql.jdbc.Driver")
         .option("user", "retail_dba")
         .option("password", "cloudera")
         .option("dbtable", "orders")
         .option("partitionColumn", "order_id")
         .option("lowerBound", "1")
         .option("upperBound", "68883")
         .option("numPartitions", "4")
         .load() 
Run Code Online (Sandbox Code Playgroud)

Question is, can I use a pseudo column (like ROWNUM in Oracle or RRN(employeeno) in DB2) with option where I specify the partitionColumn ?

If not, can we specify a partition column which is not a primary key ?

apache-spark apache-spark-sql spark-jdbc

3
推荐指数
1
解决办法
1403
查看次数