小编lef*_*oin的帖子

来自hive -h:

--hiveconf <property=value>   Use value for given property
--hivevar <key=value>         Variable subsitution to apply to hive
                                  commands. e.g. --hivevar A=B

我想在hive中将数组转换为字符串.我想collect_set数组值转换为字符串而不用[[""]].

select actor, collect_set(date) as grpdate from actor_table group by actor;

这[["2016-07-01", "2016-07-02"]]将成为2016-07-01, 2016-07-02

我遇到了气流 1.10.1 的问题。SIGTERMdag中的一些任务来自 helpers.py，据我所知，这是为工作人员执行关闭并终止所有子进程，但我只在 10 个 dag taks 中的大约 2-3 个任务中看到了这一点，并且再次运行 dag 时，接收信号的任务会发生变化。是否有一定的标准来发送这些SIGTERM信号。收到 SIGTERM 的任务的日志：

[2019-12-10 11:13:44,530] {base_task_runner.py:101} INFO - Job 25404: Subtask BS_PMU2 [2019-12-10 11:13:44,520] {settings.py:174} INFO - setting.configure_orm(): Using pool settings. pool_size=5, pool_recycle=3600
[2019-12-10 11:13:45,489] {base_task_runner.py:101} INFO - Job 25404: Subtask BS_PMU2 [2019-12-10 11:13:45,488] {__init__.py:51} INFO - Using executor CeleryExecutor
[2019-12-10 11:13:45,934] {base_task_runner.py:101} INFO - Job 25404: Subtask BS_PMU2 [2019-12-10 11:13:45,933] {models.py:271} INFO - Filling up the DagBag from /home/centos/airflow/dags/61b6c300e82643b0f294df6f.py
[2019-12-10 11:13:46,580] {base_task_runner.py:101} INFO …

运行 flutter pub run build_runner build 时出现此错误。

Failed to build build_runner:build_runner:
../../snap/flutter/common/flutter/.pub-cache/hosted/pub.dartlang.org/analyzer-0.41.2/lib/src/error/best_practices_verifier.dart:258:50: Error: The property 'displayString' is defined in multiple extensions for 'TargetKind' and neither is more specific.
 - 'TargetKind' is from 'package:meta/meta_meta.dart' ('../../snap/flutter/common/flutter/.pub-cache/hosted/pub.dartlang.org/meta-1.7.0/lib/meta_meta.dart').
Try using an explicit extension application of the wanted extension or hiding unwanted extensions from scope.
        var kindNames = kinds.map((kind) => kind.displayString).toList()
                                                 ^^^^^^^^^^^^^
../../snap/flutter/common/flutter/.pub-cache/hosted/pub.dartlang.org/analyzer-0.41.2/lib/src/error/best_practices_verifier.dart:1950:14: Context: This is one of the extension members.
  String get displayString {
             ^^^^^^^^^^^^^
../../snap/flutter/common/flutter/.pub-cache/hosted/pub.dartlang.org/meta-1.7.0/lib/meta_meta.dart:91:14: Context: This is one of the extension members. …

我有一个视图Hive 1.1.0，基于条件，它应该返回一个空数组或一个数组struct<name: string, jobslots: int>

这是我的代码：

select
      case when <condition> 
             then array()
           else array(struct(t1.name, t1.jobslots))
       end
from table t1;

这里的问题是，空数组array()的类型是array<string>。因此，当我尝试将其插入表中时，它会引发错误。

如何更改它以返回类型为空的数组，array<struct<name: string, jobslots:int>>以便Hive's size()函数在该数组上返回 0？

我正在hive中做一些自动脚本的几个查询,我们发现我们需要时间清理表中的数据并插入新的数据.我们在想什么可能更快？

INSERT OVERWRITE TABLE SOME_TABLE
    SELECT * FROM OTHER_TABLE;

或者更快这样做:

DROP TABLE SOME_TABLE;
CREATE TABLE SOME_TABLE (STUFFS);
INSERT INTO TABLE
    SELECT * FROM OTHER_TABLE;

运行查询的开销不是问题.由于我们也有创建脚本o.问题是,INSERT OVERWRITE十亿行的速度比DROP + CREATE + INSERT INTO？

我正在使用此功能来确保正确打开和关闭盒子，但我仍然收到HiveError: Box has already been closed.错误

static Future<Box<dynamic>> openIt() async {
    var connectionBox = await Hive.openBox(hiveBox);
    if (Hive.isBoxOpen(hiveBox) == false) {
      await Hive.openBox(hiveBox);
     
    } else {
      await connectionBox.close();
      connectionBox = await Hive.openBox(hiveBox);
      
    }

    return connectionBox;
  } 

我该如何解决这个问题。

如何使用json架构插入覆盖目录？

有生蜂巢avro表; (这实际上有很多领域)

tb_test--------
name string
kickname string
-----------------

然后我想通过jsonserde将查询结果保存到hdfs中的某个目录中.

我试过这个.

insert overwrite directory '/json/'
ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'
WITH SERDEPROPERTIES (
 "stat_name"="$._col0",
 "stat_interval"="$._col1"
)
STORED AS TEXTFILE 
select name, nickname
from tb_test limit 100

但在/ json /中写入json有_colXX字段名而不是原始字段名.

{"_col0":"basic_qv"," _col1":"h"}
{"_col0":"basic_qv","_col1 ":"h"}
{"_col0":"basic_qv","_col1 ":"h"}
{"_col0":"basic_qv"," _col1":"h"}
{"_col0":"basic_qv","_col1 ":"h"}

我期望

{"name":"basic_qv","nickname":"h"}
{"name":"basic_qv","nickname":"h"}
{"name":"basic_qv","nickname":"h"}
{"name":"basic_qv","nickname":"h"}
{"name":"basic_qv","nickname":"h"}

有什么用呢？

谢谢!!

我有从 JSON 文件创建的 Hive 表。

CREATE external TABLE logan_test.t1 (
   name string,
   start_time timestamp
   )
ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'
WITH SERDEPROPERTIES (
  "timestamp.formats" = "yyyy-MM-dd'T'HH:mm:ss.SSSSSS"
)
LOCATION 's3://t1/';

我的时间戳数据的格式为yyyy-MM-dd'T'HH:mm:ss.SSSSSS.

我为页面中给出的时间戳格式指定了 SERDEPROPERTIES。 https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types#LanguageManualTypes-TimestampstimestampTimestamps

创建语句成功执行但select *失败并出现以下错误。

HIVE_BAD_DATA：解析字段 1 的字段值 '2017-06-01T17:51:15.180400' 时出错：时间戳格式必须为 yyyy-mm-dd hh:mm:ss[.ffffffffff]

我正在尝试使用 hive 在 EMR 中运行的 Spark 中使用来自 Spark 的 s3 路径位置在 Glue 目录中创建一个表。我已尝试以下命令，但出现错误：

pyspark.sql.utils.AnalysisException：u'java.lang.IllegalArgumentException：无法从空字符串创建路径；'

sparksession.sql("CREATE TABLE IF NOT EXISTS abc LOCATION 's3://my-bucket/test/' as (SELECT * from my_table)")

sparksession.sql("CREATE TABLE abcSTORED AS PARQUET LOCATION 's3://my-bucket/test/' AS select * from my_table")

sparksession.sql("CREATE TABLE abcas SELECT * from my_table USING PARQUET LOCATION 's3://my-bucket/test/'")

有人可以建议我缺少的参数吗？