大查询字符串中的新行

Question

"字符串中的新行"究竟是什么意思？

https://developers.google.com/bigquery/docs/quota-policy#import

如何检查和删除它？

Import Jobs: Daily limit: 1,000 import jobs per table per day (including failures), 10,000 import jobs per project per day (including failures)
Maximum size per import job: 1TB uncompressed
Maximum number of files per import job: 500

Answer 1

字符串中的换行符是指CSV字段值中的嵌入换行符.这使得很难并行化进口.

例如:2行,第一行有一个嵌入的换行符:

1,2,"this is my
string",4,5
1,2,"another string",4,5

BigQuery导入器具有"--noallow_quoted_newlines"选项,表示该文件不包含任何嵌入的换行符.

# --noallow_quoted_newlines:
#   Do not allow quoted newlines in the data.  This allows BigQuery to parallelize the load.

https://developers.google.com/bigquery/articles/ingestioncookbook#atomic

字符串字段中没有换行符的单个JSON和CSV文件可能大到100GB,而在字符串中包含换行符的CSV文件必须为4GB或更少.

https://developers.google.com/bigquery/articles/ingestioncookbook