我尝试从保存到 HDFS 的 CSV 文件创建表。问题是 csv在引号内包含换行符。CSV 中的记录示例:
ID,PR_ID,SUMMARY
2063,1184,"This is problem field because consists line break
This is not new record but it is part of text of third column
"
Run Code Online (Sandbox Code Playgroud)
我创建了蜂巢表:
CREATE TEMPORARY EXTERNAL TABLE hive_database.hive_table
(
ID STRING,
PR_ID STRING,
SUMMARY STRING
)
row format serde 'com.bizo.hive.serde.csv.CSVSerde'
with serdeproperties (
"separatorChar" = ",",
"quoteChar" = "\"",
"escapeChar" = "\""
)
stored as textfile
LOCATION '/path/to/hdfs/dir/csv'
tblproperties('skip.header.line.count'='1');
Run Code Online (Sandbox Code Playgroud)
然后我尝试计算行数(正确的结果应该是 1)
Select count(*) from hive_database.hive_table;
Run Code Online (Sandbox Code Playgroud)
但结果是 4 what 是不正确的。你知道如何解决它吗?谢谢大家。