分区不在 Athena 上的元存储错误中

Question

我正在尝试按列对数据进行分区。但是，当我运行查询时MSCK REPAIR TABLE mytable，它返回错误

不在 Metastore 中的分区： city:countrycode=AFG city:countrycode=AGO city:countrycode=AIA city:countrycode=ALB city:countrycode=AND city:countrycode=ANT city:countrycode=ARE

我通过以下查询从 Avro 创建了表：

CREATE external table city (
  ID int,
  Name string,
  District string,
  Population int
)
PARTITIONED by (CountryCode string)
ROW FORMAT
SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
WITH SERDEPROPERTIES ('avro.schema.literal'='
{
  "fields": [
    {
      "name": "ID",
      "type": "int"
    },
    {
      "name": "Name",
      "type": "string"
    },
    {
      "name": "countrycode",
      "type": "string"
    },
    {
      "name": "District",
      "type": "string"
    },
    {
      "name": "Population",
      "type": "int"
    }
  ],
  "name": "value",
  "namespace": "world.city",
  "type": "record"
}
')
STORED AS AVRO
LOCATION "s3://mybucket/city"

我的分区看起来像s3://mybucket/city/countrycode=ABC

Answer 1

您的位置肯定需要尾部斜杠： https://docs.aws.amazon.com/athena/latest/ug/create-table.html

也许也可以尝试使用小写的分区列PARTITIONED by (countrycode string)。

您是否尝试在 Glue Catalog 中或通过 Crawler 手动添加分区？这有效吗？